2024-05-14T16:40:15.263 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-14T16:40:15.263 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182 --name leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} -- /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/orig.config.yaml 2024-05-14T16:40:15.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T16:40:15.430 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182 branch: wip-lusov-quiesce description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7706182' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 710 sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiesce suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBiw6ukUsbpFXoMLy1it/jvYB0cNHolqhaDjCzxMXmctvBbEDA3k22Et5mgpxYNkmHJt8KTUH6A6OBjv/UficvI= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFnPtW/ydXNhKazr9+bslerCnNDZ5m0xnt08qKhiVkvrSm3V2YUkJdQZklyFK/ElaJDaEuFwwBpQSfc84DzK6mk= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ThY9+JfMC+UiF7NZfMnkvWr82I2LnuWBPu2xxg+HGJQTXUm953LGwGKC9NOe7SX7shQkPL15XJ+rsvh0BTiI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-14_16:11:40 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-14T16:40:15.430 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa; will attempt to use it 2024-05-14T16:40:15.431 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks 2024-05-14T16:40:15.431 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-14T16:40:15.432 INFO:teuthology.task.internal:Checking packages... 2024-05-14T16:40:15.457 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b' 2024-05-14T16:40:15.457 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-14T16:40:15.457 INFO:teuthology.packaging:ref: None 2024-05-14T16:40:15.457 INFO:teuthology.packaging:tag: None 2024-05-14T16:40:15.457 INFO:teuthology.packaging:branch: wip-lusov-quiesce 2024-05-14T16:40:15.457 INFO:teuthology.packaging:sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:40:15.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce 2024-05-14T16:40:15.628 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3687.g9b638668 2024-05-14T16:40:15.630 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-14T16:40:15.637 INFO:teuthology.task.internal:no buildpackages task found 2024-05-14T16:40:15.637 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-14T16:40:15.645 INFO:teuthology.task.internal:Saving configuration 2024-05-14T16:40:15.667 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-14T16:40:15.673 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T16:40:15.699 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:34:57.919365', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBiw6ukUsbpFXoMLy1it/jvYB0cNHolqhaDjCzxMXmctvBbEDA3k22Et5mgpxYNkmHJt8KTUH6A6OBjv/UficvI='} 2024-05-14T16:40:15.720 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:34:57.920400', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFnPtW/ydXNhKazr9+bslerCnNDZ5m0xnt08qKhiVkvrSm3V2YUkJdQZklyFK/ElaJDaEuFwwBpQSfc84DzK6mk='} 2024-05-14T16:40:15.746 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:34:57.917640', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ThY9+JfMC+UiF7NZfMnkvWr82I2LnuWBPu2xxg+HGJQTXUm953LGwGKC9NOe7SX7shQkPL15XJ+rsvh0BTiI='} 2024-05-14T16:40:15.746 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-14T16:40:15.753 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-14T16:40:15.753 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-14T16:40:15.753 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-14T16:40:15.753 INFO:teuthology.run_tasks:Running task console_log... 2024-05-14T16:40:15.858 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6262d5f0d0>, signals=[15]) 2024-05-14T16:40:15.859 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-14T16:40:15.865 INFO:teuthology.task.internal:Opening connections... 2024-05-14T16:40:15.865 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T16:40:15.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:40:15.949 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-05-14T16:40:15.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:40:16.021 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-05-14T16:40:16.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:40:16.095 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-14T16:40:16.103 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-05-14T16:40:16.121 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-05-14T16:40:16.121 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:40:16.178 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:40:16.179 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:40:16.179 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:40:16.179 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-05-14T16:40:16.203 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-05-14T16:40:16.221 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-05-14T16:40:16.221 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:40:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:40:16.279 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:40:16.279 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-05-14T16:40:16.303 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-05-14T16:40:16.320 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-05-14T16:40:16.321 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:40:16.378 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:40:16.379 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:40:16.379 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:40:16.379 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:40:16.379 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:40:16.379 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-05-14T16:40:16.630 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-14T16:40:16.642 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-14T16:40:16.649 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-14T16:40:16.649 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:40:16.651 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:40:16.653 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:40:16.670 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-14T16:40:16.676 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-14T16:40:16.676 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:40:16.708 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:40:16.711 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:40:16.741 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-14T16:40:16.817 INFO:teuthology.run_tasks:Running task kernel... 2024-05-14T16:40:16.831 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-14T16:40:16.831 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-14T16:40:16.831 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-14T16:40:16.832 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:40:16.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:40:16.848 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-05-14T16:40:16.906 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-437.el9.x86_64 2024-05-14T16:40:16.906 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.14.0-437.el9.x86_64 2024-05-14T16:40:16.906 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2024-05-14T16:40:19.245 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-14T16:40:22.754 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-14T16:40:26.804 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-05-14T16:40:28.654 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-14T16:40:29.750 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-14T16:40:34.931 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-14T16:40:36.912 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-14T16:40:36.913 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-14T16:40:37.006 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-05-14T16:40:37.015 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout:Install 4 Packages 2024-05-14T16:40:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:40:37.017 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 78 M 2024-05-14T16:40:37.017 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 123 M 2024-05-14T16:40:37.017 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-05-14T16:40:38.065 INFO:teuthology.orchestra.run.smithi099.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 560 kB/s | 550 kB 00:00 2024-05-14T16:40:38.665 INFO:teuthology.orchestra.run.smithi099.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 9.8 MB/s | 15 MB 00:01 2024-05-14T16:40:39.457 INFO:teuthology.orchestra.run.smithi099.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-05-14T16:40:41.257 INFO:teuthology.orchestra.run.smithi099.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 8.8 MB/s | 28 MB 00:03 2024-05-14T16:40:41.258 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:40:41.258 INFO:teuthology.orchestra.run.smithi099.stdout:Total 18 MB/s | 78 MB 00:04 2024-05-14T16:40:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-05-14T16:40:42.301 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-05-14T16:40:42.301 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-05-14T16:40:43.210 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-05-14T16:40:43.210 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-05-14T16:40:45.587 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-05-14T16:40:46.269 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-14T16:40:46.358 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:40:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:40:49.148 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:40:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:40:53.859 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:40:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:41:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:41:27.923 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:41:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:41:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-14T16:41:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:41:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:41:29.053 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:41:29.053 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:41:29.054 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-05-14T16:41:29.412 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T16:41:29.490 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T16:41:29.490 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T16:41:29.491 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-14T16:41:29.491 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-14T16:41:29.491 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-14T16:41:29.491 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-14T16:41:29.491 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2024-05-14T16:41:29.491 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T16:41:29.541 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T16:41:29.542 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T16:41:29.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-05-14T16:41:29.579 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-05-14T16:41:29.579 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-05-14T16:41:29.579 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 68.el9 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8264254 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2024-05-14T16:41:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:41:29.581 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-05-14T16:41:29.582 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-14T16:41:29.582 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 5.14.0-446.el9.x86_64 2024-05-14T16:41:29.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-14T16:41:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-05-14T16:41:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-14T16:41:31.645 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-05-14T16:41:31.656 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-05-14T16:41:31.677 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.j79a7SRVEO 2024-05-14T16:41:31.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.j79a7SRVEO 2024-05-14T16:41:31.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.j79a7SRVEO 2024-05-14T16:41:31.892 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.j79a7SRVEO is 6KB 2024-05-14T16:41:31.941 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.j79a7SRVEO 2024-05-14T16:41:31.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-05-14T16:41:32.018 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-14T16:41:32.018 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-14T16:41:32.018 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-14T16:41:32.018 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-14T16:41:32.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-14T16:41:32.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-05-14T16:42:02.148 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-14T16:42:02.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-14T16:42:02.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:43:02.152 DEBUG:teuthology.orchestra.remote:timed out 2024-05-14T16:43:11.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-14T16:43:11.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:43:14.583 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-05-14T16:43:14.986 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-14T16:43:14.986 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-14T16:43:14.987 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-05-14T16:43:15.001 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-446.el9.x86_64 2024-05-14T16:43:15.001 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-14T16:43:15.002 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T16:43:15.002 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-14T16:43:16.003 DEBUG:teuthology.parallel:result is None 2024-05-14T16:43:16.003 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-14T16:43:16.010 INFO:teuthology.task.internal:Creating test directory... 2024-05-14T16:43:16.010 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:43:16.012 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:43:16.015 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:43:16.032 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-14T16:43:16.039 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-14T16:43:16.049 INFO:teuthology.task.internal:Creating archive directory... 2024-05-14T16:43:16.049 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:43:16.068 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:43:16.073 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:43:16.111 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-14T16:43:16.117 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-14T16:43:16.117 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:43:16.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:43:16.147 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:43:16.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:43:16.164 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:43:16.166 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:43:16.189 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:43:16.240 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:43:16.242 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:43:16.242 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:43:16.262 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:43:16.264 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:43:16.264 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:43:16.265 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-14T16:43:16.273 INFO:teuthology.task.internal:Configuring sudo... 2024-05-14T16:43:16.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:43:16.275 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:43:16.277 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:43:16.331 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-14T16:43:16.340 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-14T16:43:16.341 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:43:16.343 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:43:16.345 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:43:16.387 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:43:16.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:43:16.482 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:43:16.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:43:16.605 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:43:16.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:43:16.684 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:43:16.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:43:16.781 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:43:16.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:43:16.904 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:43:16.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:43:16.969 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:43:17.015 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:43:17.070 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:43:17.156 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:43:17.196 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:43:17.196 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:43:17.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-05-14T16:43:17.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-05-14T16:43:17.267 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-05-14T16:43:17.302 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:43:17.306 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:43:17.342 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:43:17.830 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-14T16:43:17.838 INFO:teuthology.task.internal:Starting timer... 2024-05-14T16:43:17.838 INFO:teuthology.run_tasks:Running task pcp... 2024-05-14T16:43:17.880 INFO:teuthology.run_tasks:Running task selinux... 2024-05-14T16:43:17.888 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-14T16:43:17.888 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-05-14T16:43:17.952 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-05-14T16:43:17.954 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-05-14T16:43:18.035 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-05-14T16:43:18.037 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-05-14T16:43:18.106 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-05-14T16:43:18.108 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-14T16:43:18.108 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-05-14T16:43:18.131 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-05-14T16:43:18.131 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-05-14T16:43:18.158 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-05-14T16:43:18.158 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-05-14T16:43:18.178 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-05-14T16:43:18.179 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-05-14T16:43:18.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:43:18.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:43:18.204 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:43:18.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:43:18.231 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:43:18.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:43:18.260 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-14T16:43:18.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:43:18.291 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:43:18.316 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:43:18.348 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-14T16:43:18.358 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-14T16:43:18.364 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-14T16:43:18.522 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-14T16:43:18.541 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-14T16:43:18.544 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-14T16:49:10.354 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-05-14T16:49:10.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-14T16:49:10.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:49:10.437 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-05-14T16:49:10.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-14T16:49:10.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-14T16:49:10.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:49:10.587 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-05-14T16:49:10.661 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-14T16:49:10.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-14T16:49:10.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:49:10.780 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-05-14T16:49:10.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-14T16:49:10.866 INFO:teuthology.run_tasks:Running task clock... 2024-05-14T16:49:10.876 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-14T16:49:10.877 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:49:10.877 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:49:10.879 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:49:10.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:49:10.882 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:49:10.882 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:49:10.908 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:49:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:49:10.922 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:49:10.925 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:49:10.936 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:49:10.944 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-05-14T16:49:10.949 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:49:10.950 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-05-14T16:49:10.954 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-05-14T16:49:10.961 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-05-14T16:49:10.967 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:49:10.973 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-05-14T16:49:10.975 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:49:10.980 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:49:10.983 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-05-14T16:49:10.990 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:49:10.996 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:49:11.009 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:49:11.037 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:49:11.040 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:49:11.040 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-14T16:49:11.040 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.040 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.040 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.040 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.042 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:49:11.045 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:49:11.045 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-05-14T16:49:11.046 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.046 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.046 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.046 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.063 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:49:11.067 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:49:11.067 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-05-14T16:49:11.067 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.067 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.067 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.067 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:49:11.068 INFO:teuthology.run_tasks:Running task install... 2024-05-14T16:49:11.076 DEBUG:teuthology.task.install:project ceph 2024-05-14T16:49:11.076 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'}} 2024-05-14T16:49:11.076 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'} 2024-05-14T16:49:11.076 INFO:teuthology.task.install:Using flavor: default 2024-05-14T16:49:11.082 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-14T16:49:11.082 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-14T16:49:11.082 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:49:11.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:49:11.085 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:49:11.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:49:11.086 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:49:11.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:49:11.235 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:49:11.236 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:49:11.241 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:49:11.241 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:49:11.298 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:49:11.298 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:49:11.408 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:49:11.408 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:49:11.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:49:11.414 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:49:11.414 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:49:11.414 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:49:11.435 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:49:11.436 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:49:11.439 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:49:11.440 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:49:11.466 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:49:11.466 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:49:11.466 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:49:11.492 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:49:11.492 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:49:11.501 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:49:11.504 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:49:11.558 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:49:11.575 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:49:11.577 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:49:11.632 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:49:11.639 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-05-14T16:49:11.640 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-05-14T16:49:11.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-05-14T16:49:11.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-05-14T16:49:11.695 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-05-14T16:49:11.697 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-05-14T16:49:11.872 INFO:teuthology.orchestra.run.smithi150.stdout:57 files removed 2024-05-14T16:49:11.877 INFO:teuthology.orchestra.run.smithi099.stdout:57 files removed 2024-05-14T16:49:11.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:49:11.904 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:49:11.928 INFO:teuthology.orchestra.run.smithi193.stdout:57 files removed 2024-05-14T16:49:11.955 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:49:12.582 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 198 kB/s | 84 kB 00:00 2024-05-14T16:49:12.588 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-05-14T16:49:12.683 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 179 kB/s | 84 kB 00:00 2024-05-14T16:49:12.947 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-05-14T16:49:13.007 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-14T16:49:13.133 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-14T16:49:13.381 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-14T16:49:13.416 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-14T16:49:13.517 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-14T16:49:13.986 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.1 MB 00:00 2024-05-14T16:49:14.983 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.1 MB 00:01 2024-05-14T16:49:15.048 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.1 MB 00:01 2024-05-14T16:49:16.676 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-05-14T16:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-14T16:49:18.719 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-05-14T16:49:20.918 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-14T16:49:21.802 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-05-14T16:49:22.453 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-05-14T16:49:24.394 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-14T16:49:24.660 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 9.6 kB/s | 16 kB 00:01 2024-05-14T16:49:24.907 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 21 MB 00:02 2024-05-14T16:49:25.812 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-05-14T16:49:30.146 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-14T16:49:30.939 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-05-14T16:49:32.388 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:49:32.389 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:49:32.399 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:49:32.400 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:49:32.401 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:49:32.402 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:49:32.403 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:49:32.405 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:49:32.407 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:49:32.466 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-14T16:49:32.480 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-14T16:49:32.480 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-14T16:49:32.480 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-14T16:49:32.480 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-14T16:49:32.480 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:49:32.480 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:49:32.481 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:49:32.482 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:49:32.483 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:49:32.484 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:49:32.485 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:49:32.486 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:49:32.487 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:49:32.488 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:49:32.489 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:49:32.490 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:49:32.491 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:49:32.492 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:49:32.493 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:49:32.494 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-05-14T16:49:32.495 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:49:32.496 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:49:32.496 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-14T16:49:32.496 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-14T16:49:32.496 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-05-14T16:49:32.496 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 8 Packages 2024-05-14T16:49:32.496 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:49:32.497 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 226 M 2024-05-14T16:49:32.497 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-14T16:49:33.155 INFO:teuthology.orchestra.run.smithi193.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-05-14T16:49:33.339 INFO:teuthology.orchestra.run.smithi193.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 4.7 MB/s | 891 kB 00:00 2024-05-14T16:49:33.389 INFO:teuthology.orchestra.run.smithi193.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 2.8 MB/s | 145 kB 00:00 2024-05-14T16:49:33.739 INFO:teuthology.orchestra.run.smithi193.stdout:(4/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 6.9 MB/s | 5.4 MB 00:00 2024-05-14T16:49:33.790 INFO:teuthology.orchestra.run.smithi193.stdout:(5/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-05-14T16:49:33.923 INFO:teuthology.orchestra.run.smithi193.stdout:(6/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-05-14T16:49:34.357 INFO:teuthology.orchestra.run.smithi193.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-05-14T16:49:35.365 INFO:teuthology.orchestra.run.smithi193.stdout:(8/156): ceph-osd-19.0.0-3687.g9b638668.el9.x86 12 MB/s | 17 MB 00:01 2024-05-14T16:49:35.416 INFO:teuthology.orchestra.run.smithi193.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 504 kB/s | 25 kB 00:00 2024-05-14T16:49:35.866 INFO:teuthology.orchestra.run.smithi193.stdout:(10/156): ceph-radosgw-19.0.0-3687.g9b638668.el 7.0 MB/s | 11 MB 00:01 2024-05-14T16:49:35.941 INFO:teuthology.orchestra.run.smithi193.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 2.9 MB/s | 225 kB 00:00 2024-05-14T16:49:36.008 INFO:teuthology.orchestra.run.smithi193.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 405 kB/s | 27 kB 00:00 2024-05-14T16:49:36.128 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-14T16:49:36.234 INFO:teuthology.orchestra.run.smithi193.stdout:(13/156): ceph-common-19.0.0-3687.g9b638668.el9 6.5 MB/s | 21 MB 00:03 2024-05-14T16:49:36.259 INFO:teuthology.orchestra.run.smithi193.stdout:(14/156): libcephfs-devel-19.0.0-3687.g9b638668 127 kB/s | 32 kB 00:00 2024-05-14T16:49:36.334 INFO:teuthology.orchestra.run.smithi193.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 2.1 MB/s | 163 kB 00:00 2024-05-14T16:49:36.393 INFO:teuthology.orchestra.run.smithi193.stdout:(16/156): libcephfs2-19.0.0-3687.g9b638668.el9. 4.6 MB/s | 748 kB 00:00 2024-05-14T16:49:36.418 INFO:teuthology.orchestra.run.smithi193.stdout:(17/156): librados-devel-19.0.0-3687.g9b638668. 1.5 MB/s | 127 kB 00:00 2024-05-14T16:49:36.535 INFO:teuthology.orchestra.run.smithi193.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 3.4 MB/s | 501 kB 00:00 2024-05-14T16:49:36.619 INFO:teuthology.orchestra.run.smithi193.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 542 kB/s | 45 kB 00:00 2024-05-14T16:49:36.702 INFO:teuthology.orchestra.run.smithi193.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.6 MB/s | 136 kB 00:00 2024-05-14T16:49:36.794 INFO:teuthology.orchestra.run.smithi193.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.7 MB/s | 161 kB 00:00 2024-05-14T16:49:36.894 INFO:teuthology.orchestra.run.smithi193.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 3.1 MB/s | 321 kB 00:00 2024-05-14T16:49:37.011 INFO:teuthology.orchestra.run.smithi193.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 2.5 MB/s | 299 kB 00:00 2024-05-14T16:49:37.170 INFO:teuthology.orchestra.run.smithi193.stdout:(24/156): librgw2-19.0.0-3687.g9b638668.el9.x86 7.1 MB/s | 5.4 MB 00:00 2024-05-14T16:49:37.195 INFO:teuthology.orchestra.run.smithi193.stdout:(25/156): python3-rgw-19.0.0-3687.g9b638668.el9 542 kB/s | 100 kB 00:00 2024-05-14T16:49:37.237 INFO:teuthology.orchestra.run.smithi193.stdout:(26/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-14T16:49:37.312 INFO:teuthology.orchestra.run.smithi193.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-14T16:49:37.396 INFO:teuthology.orchestra.run.smithi193.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 411 kB/s | 34 kB 00:00 2024-05-14T16:49:37.488 INFO:teuthology.orchestra.run.smithi193.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.5 MB/s | 143 kB 00:00 2024-05-14T16:49:37.747 INFO:teuthology.orchestra.run.smithi193.stdout:(30/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 5.6 MB/s | 3.1 MB 00:00 2024-05-14T16:49:38.097 INFO:teuthology.orchestra.run.smithi193.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 5.8 MB/s | 3.5 MB 00:00 2024-05-14T16:49:38.421 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:49:38.421 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:49:38.432 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:49:38.433 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:49:38.434 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:49:38.434 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:49:38.436 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:49:38.438 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:49:38.439 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:49:38.500 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:49:38.513 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:49:38.514 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:49:38.515 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:49:38.516 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:49:38.517 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:49:38.518 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:49:38.519 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:49:38.520 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:49:38.521 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:49:38.522 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:49:38.523 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:49:38.524 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 8 Packages 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 226 M 2024-05-14T16:49:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-14T16:49:38.581 INFO:teuthology.orchestra.run.smithi193.stdout:(32/156): ceph-test-19.0.0-3687.g9b638668.el9.x 16 MB/s | 49 MB 00:03 2024-05-14T16:49:38.606 INFO:teuthology.orchestra.run.smithi193.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3687.g9b 540 kB/s | 274 kB 00:00 2024-05-14T16:49:38.631 INFO:teuthology.orchestra.run.smithi193.stdout:(34/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 992 kB/s | 49 kB 00:00 2024-05-14T16:49:38.656 INFO:teuthology.orchestra.run.smithi193.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3687.g9 328 kB/s | 16 kB 00:00 2024-05-14T16:49:38.681 INFO:teuthology.orchestra.run.smithi193.stdout:(36/156): ceph-volume-19.0.0-3687.g9b638668.el9 5.3 MB/s | 273 kB 00:00 2024-05-14T16:49:38.790 INFO:teuthology.orchestra.run.smithi193.stdout:(37/156): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-05-14T16:49:38.815 INFO:teuthology.orchestra.run.smithi193.stdout:(38/156): cephfs-top-19.0.0-3687.g9b638668.el9. 136 kB/s | 18 kB 00:00 2024-05-14T16:49:38.841 INFO:teuthology.orchestra.run.smithi193.stdout:(39/156): cephadm-19.0.0-3687.g9b638668.el9.noa 4.1 MB/s | 763 kB 00:00 2024-05-14T16:49:38.933 INFO:teuthology.orchestra.run.smithi193.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-05-14T16:49:39.000 INFO:teuthology.orchestra.run.smithi193.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-14T16:49:39.033 INFO:teuthology.orchestra.run.smithi193.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-14T16:49:39.059 INFO:teuthology.orchestra.run.smithi193.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-05-14T16:49:39.084 INFO:teuthology.orchestra.run.smithi193.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-14T16:49:39.109 INFO:teuthology.orchestra.run.smithi193.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-14T16:49:39.134 INFO:teuthology.orchestra.run.smithi193.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-14T16:49:39.160 INFO:teuthology.orchestra.run.smithi193.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-14T16:49:39.193 INFO:teuthology.orchestra.run.smithi193.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-05-14T16:49:39.429 INFO:teuthology.orchestra.run.smithi150.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-05-14T16:49:39.743 INFO:teuthology.orchestra.run.smithi193.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 194 kB/s | 107 kB 00:00 2024-05-14T16:49:39.805 INFO:teuthology.orchestra.run.smithi150.stdout:(2/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 7.6 MB/s | 5.4 MB 00:00 2024-05-14T16:49:39.863 INFO:teuthology.orchestra.run.smithi150.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 2.4 MB/s | 145 kB 00:00 2024-05-14T16:49:39.877 INFO:teuthology.orchestra.run.smithi193.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-05-14T16:49:40.072 INFO:teuthology.orchestra.run.smithi150.stdout:(4/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 1.4 MB/s | 891 kB 00:00 2024-05-14T16:49:40.128 INFO:teuthology.orchestra.run.smithi193.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 694 kB/s | 689 kB 00:00 2024-05-14T16:49:40.147 INFO:teuthology.orchestra.run.smithi150.stdout:(5/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-05-14T16:49:40.161 INFO:teuthology.orchestra.run.smithi193.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 942 kB/s | 943 kB 00:01 2024-05-14T16:49:40.270 INFO:teuthology.orchestra.run.smithi193.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-05-14T16:49:40.412 INFO:teuthology.orchestra.run.smithi193.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 18 kB 00:00 2024-05-14T16:49:40.512 INFO:teuthology.orchestra.run.smithi193.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-14T16:49:40.620 INFO:teuthology.orchestra.run.smithi193.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-14T16:49:40.648 INFO:teuthology.orchestra.run.smithi150.stdout:(6/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-05-14T16:49:40.720 INFO:teuthology.orchestra.run.smithi193.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.3 MB/s | 3.0 MB 00:00 2024-05-14T16:49:40.746 INFO:teuthology.orchestra.run.smithi193.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-05-14T16:49:40.829 INFO:teuthology.orchestra.run.smithi193.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-14T16:49:40.865 INFO:teuthology.orchestra.run.smithi150.stdout:(7/156): ceph-common-19.0.0-3687.g9b638668.el9. 12 MB/s | 21 MB 00:01 2024-05-14T16:49:40.898 INFO:teuthology.orchestra.run.smithi150.stdout:(8/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-05-14T16:49:40.930 INFO:teuthology.orchestra.run.smithi193.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-05-14T16:49:40.990 INFO:teuthology.orchestra.run.smithi150.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 274 kB/s | 25 kB 00:00 2024-05-14T16:49:41.013 INFO:teuthology.orchestra.run.smithi193.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-05-14T16:49:41.039 INFO:teuthology.orchestra.run.smithi193.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-05-14T16:49:41.122 INFO:teuthology.orchestra.run.smithi193.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-14T16:49:41.148 INFO:teuthology.orchestra.run.smithi193.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-14T16:49:41.223 INFO:teuthology.orchestra.run.smithi193.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-14T16:49:41.273 INFO:teuthology.orchestra.run.smithi193.stdout:(66/156): flex-2.6.4-9.el9.x86_64.rpm 225 kB/s | 313 kB 00:01 2024-05-14T16:49:41.607 INFO:teuthology.orchestra.run.smithi150.stdout:(10/156): ceph-radosgw-19.0.0-3687.g9b638668.el 14 MB/s | 11 MB 00:00 2024-05-14T16:49:41.643 INFO:teuthology.orchestra.run.smithi193.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-14T16:49:41.691 INFO:teuthology.orchestra.run.smithi150.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 2.6 MB/s | 225 kB 00:00 2024-05-14T16:49:41.757 INFO:teuthology.orchestra.run.smithi193.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-14T16:49:41.757 INFO:teuthology.orchestra.run.smithi150.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 406 kB/s | 27 kB 00:00 2024-05-14T16:49:41.824 INFO:teuthology.orchestra.run.smithi150.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 475 kB/s | 32 kB 00:00 2024-05-14T16:49:41.865 INFO:teuthology.orchestra.run.smithi193.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-14T16:49:41.916 INFO:teuthology.orchestra.run.smithi150.stdout:(14/156): libcephfs2-19.0.0-3687.g9b638668.el9. 8.0 MB/s | 748 kB 00:00 2024-05-14T16:49:41.975 INFO:teuthology.orchestra.run.smithi150.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 2.7 MB/s | 163 kB 00:00 2024-05-14T16:49:42.142 INFO:teuthology.orchestra.run.smithi150.stdout:(16/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 11 MB/s | 17 MB 00:01 2024-05-14T16:49:42.167 INFO:teuthology.orchestra.run.smithi150.stdout:(17/156): librados-devel-19.0.0-3687.g9b638668. 659 kB/s | 127 kB 00:00 2024-05-14T16:49:42.201 INFO:teuthology.orchestra.run.smithi150.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 8.4 MB/s | 501 kB 00:00 2024-05-14T16:49:42.268 INFO:teuthology.orchestra.run.smithi150.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 678 kB/s | 45 kB 00:00 2024-05-14T16:49:42.343 INFO:teuthology.orchestra.run.smithi150.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.8 MB/s | 136 kB 00:00 2024-05-14T16:49:42.410 INFO:teuthology.orchestra.run.smithi150.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 2.4 MB/s | 161 kB 00:00 2024-05-14T16:49:42.485 INFO:teuthology.orchestra.run.smithi150.stdout:(22/156): librgw2-19.0.0-3687.g9b638668.el9.x86 17 MB/s | 5.4 MB 00:00 2024-05-14T16:49:42.544 INFO:teuthology.orchestra.run.smithi150.stdout:(23/156): python3-rados-19.0.0-3687.g9b638668.e 3.1 MB/s | 321 kB 00:00 2024-05-14T16:49:42.544 INFO:teuthology.orchestra.run.smithi150.stdout:(24/156): python3-rbd-19.0.0-3687.g9b638668.el9 5.0 MB/s | 299 kB 00:00 2024-05-14T16:49:42.569 INFO:teuthology.orchestra.run.smithi150.stdout:(25/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.7 MB/s | 100 kB 00:00 2024-05-14T16:49:42.595 INFO:teuthology.orchestra.run.smithi150.stdout:(26/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-14T16:49:42.662 INFO:teuthology.orchestra.run.smithi150.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-14T16:49:42.729 INFO:teuthology.orchestra.run.smithi150.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 511 kB/s | 34 kB 00:00 2024-05-14T16:49:42.779 INFO:teuthology.orchestra.run.smithi150.stdout:(29/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 15 MB/s | 3.1 MB 00:00 2024-05-14T16:49:42.804 INFO:teuthology.orchestra.run.smithi150.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.9 MB/s | 143 kB 00:00 2024-05-14T16:49:42.832 INFO:teuthology.orchestra.run.smithi193.stdout:(70/156): postgresql-13.14-1.el9.x86_64.rpm 1.0 MB/s | 1.6 MB 00:01 2024-05-14T16:49:43.155 INFO:teuthology.orchestra.run.smithi150.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 9.4 MB/s | 3.5 MB 00:00 2024-05-14T16:49:43.238 INFO:teuthology.orchestra.run.smithi150.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 3.2 MB/s | 274 kB 00:00 2024-05-14T16:49:43.331 INFO:teuthology.orchestra.run.smithi150.stdout:(33/156): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-05-14T16:49:43.356 INFO:teuthology.orchestra.run.smithi150.stdout:(34/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 421 kB/s | 49 kB 00:00 2024-05-14T16:49:43.381 INFO:teuthology.orchestra.run.smithi150.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3687.g9 328 kB/s | 16 kB 00:00 2024-05-14T16:49:43.406 INFO:teuthology.orchestra.run.smithi150.stdout:(36/156): ceph-volume-19.0.0-3687.g9b638668.el9 5.3 MB/s | 273 kB 00:00 2024-05-14T16:49:43.448 INFO:teuthology.orchestra.run.smithi150.stdout:(37/156): cephadm-19.0.0-3687.g9b638668.el9.noa 11 MB/s | 763 kB 00:00 2024-05-14T16:49:43.473 INFO:teuthology.orchestra.run.smithi150.stdout:(38/156): cephfs-top-19.0.0-3687.g9b638668.el9. 273 kB/s | 18 kB 00:00 2024-05-14T16:49:43.532 INFO:teuthology.orchestra.run.smithi150.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 961 kB/s | 80 kB 00:00 2024-05-14T16:49:43.583 INFO:teuthology.orchestra.run.smithi150.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-05-14T16:49:43.583 INFO:teuthology.orchestra.run.smithi193.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-05-14T16:49:43.616 INFO:teuthology.orchestra.run.smithi150.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-14T16:49:43.658 INFO:teuthology.orchestra.run.smithi193.stdout:(72/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.9 MB/s | 4.7 MB 00:02 2024-05-14T16:49:43.666 INFO:teuthology.orchestra.run.smithi150.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-14T16:49:43.675 INFO:teuthology.orchestra.run.smithi193.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 950 kB/s | 87 kB 00:00 2024-05-14T16:49:43.733 INFO:teuthology.orchestra.run.smithi150.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-14T16:49:43.767 INFO:teuthology.orchestra.run.smithi150.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-05-14T16:49:43.817 INFO:teuthology.orchestra.run.smithi150.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-14T16:49:43.851 INFO:teuthology.orchestra.run.smithi150.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-14T16:49:43.858 INFO:teuthology.orchestra.run.smithi193.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-05-14T16:49:43.993 INFO:teuthology.orchestra.run.smithi150.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.2 MB/s | 215 kB 00:00 2024-05-14T16:49:44.042 INFO:teuthology.orchestra.run.smithi193.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-05-14T16:49:44.118 INFO:teuthology.orchestra.run.smithi150.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-14T16:49:44.134 INFO:teuthology.orchestra.run.smithi193.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-05-14T16:49:44.168 INFO:teuthology.orchestra.run.smithi150.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-14T16:49:44.211 INFO:teuthology.orchestra.run.smithi150.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-05-14T16:49:44.226 INFO:teuthology.orchestra.run.smithi193.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-05-14T16:49:44.236 INFO:teuthology.orchestra.run.smithi150.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-14T16:49:44.261 INFO:teuthology.orchestra.run.smithi150.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-14T16:49:44.294 INFO:teuthology.orchestra.run.smithi150.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-05-14T16:49:44.335 INFO:teuthology.orchestra.run.smithi193.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-14T16:49:44.395 INFO:teuthology.orchestra.run.smithi150.stdout:(54/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-05-14T16:49:44.428 INFO:teuthology.orchestra.run.smithi150.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-05-14T16:49:44.435 INFO:teuthology.orchestra.run.smithi193.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-05-14T16:49:44.471 INFO:teuthology.orchestra.run.smithi150.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 147 kB/s | 11 kB 00:00 2024-05-14T16:49:44.496 INFO:teuthology.orchestra.run.smithi150.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-05-14T16:49:44.546 INFO:teuthology.orchestra.run.smithi150.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-05-14T16:49:44.572 INFO:teuthology.orchestra.run.smithi150.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-05-14T16:49:44.605 INFO:teuthology.orchestra.run.smithi150.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-14T16:49:44.622 INFO:teuthology.orchestra.run.smithi150.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-14T16:49:44.639 INFO:teuthology.orchestra.run.smithi150.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-14T16:49:44.673 INFO:teuthology.orchestra.run.smithi150.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-14T16:49:44.698 INFO:teuthology.orchestra.run.smithi150.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-14T16:49:44.723 INFO:teuthology.orchestra.run.smithi150.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-14T16:49:44.824 INFO:teuthology.orchestra.run.smithi150.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-05-14T16:49:44.852 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-05-14T16:49:44.899 INFO:teuthology.orchestra.run.smithi150.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-14T16:49:44.941 INFO:teuthology.orchestra.run.smithi150.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-14T16:49:44.991 INFO:teuthology.orchestra.run.smithi150.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-05-14T16:49:45.017 INFO:teuthology.orchestra.run.smithi150.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-05-14T16:49:45.084 INFO:teuthology.orchestra.run.smithi150.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-14T16:49:45.134 INFO:teuthology.orchestra.run.smithi150.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-14T16:49:45.226 INFO:teuthology.orchestra.run.smithi150.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-05-14T16:49:45.276 INFO:teuthology.orchestra.run.smithi150.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-14T16:49:45.326 INFO:teuthology.orchestra.run.smithi150.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-14T16:49:45.368 INFO:teuthology.orchestra.run.smithi150.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-05-14T16:49:45.452 INFO:teuthology.orchestra.run.smithi150.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-05-14T16:49:45.477 INFO:teuthology.orchestra.run.smithi150.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-05-14T16:49:45.519 INFO:teuthology.orchestra.run.smithi150.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-05-14T16:49:45.736 INFO:teuthology.orchestra.run.smithi150.stdout:(80/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-05-14T16:49:45.786 INFO:teuthology.orchestra.run.smithi150.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-14T16:49:45.811 INFO:teuthology.orchestra.run.smithi150.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 479 kB/s | 172 kB 00:00 2024-05-14T16:49:45.837 INFO:teuthology.orchestra.run.smithi150.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-14T16:49:45.862 INFO:teuthology.orchestra.run.smithi150.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-14T16:49:45.879 INFO:teuthology.orchestra.run.smithi150.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-14T16:49:45.913 INFO:teuthology.orchestra.run.smithi150.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-14T16:49:45.938 INFO:teuthology.orchestra.run.smithi150.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-05-14T16:49:45.964 INFO:teuthology.orchestra.run.smithi150.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-14T16:49:45.989 INFO:teuthology.orchestra.run.smithi150.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-14T16:49:46.022 INFO:teuthology.orchestra.run.smithi150.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 920 kB/s | 54 kB 00:00 2024-05-14T16:49:46.064 INFO:teuthology.orchestra.run.smithi150.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 999 kB/s | 42 kB 00:00 2024-05-14T16:49:46.115 INFO:teuthology.orchestra.run.smithi150.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-05-14T16:49:46.165 INFO:teuthology.orchestra.run.smithi150.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-14T16:49:46.216 INFO:teuthology.orchestra.run.smithi150.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-05-14T16:49:46.374 INFO:teuthology.orchestra.run.smithi150.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-05-14T16:49:46.399 INFO:teuthology.orchestra.run.smithi150.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-05-14T16:49:46.425 INFO:teuthology.orchestra.run.smithi150.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-14T16:49:46.475 INFO:teuthology.orchestra.run.smithi150.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-14T16:49:46.502 INFO:teuthology.orchestra.run.smithi193.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 1.2 MB/s | 5.8 MB 00:04 2024-05-14T16:49:46.525 INFO:teuthology.orchestra.run.smithi150.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-14T16:49:46.576 INFO:teuthology.orchestra.run.smithi150.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-05-14T16:49:46.618 INFO:teuthology.orchestra.run.smithi193.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-14T16:49:46.727 INFO:teuthology.orchestra.run.smithi193.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-14T16:49:46.868 INFO:teuthology.orchestra.run.smithi150.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-05-14T16:49:46.877 INFO:teuthology.orchestra.run.smithi193.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 515 kB/s | 77 kB 00:00 2024-05-14T16:49:46.944 INFO:teuthology.orchestra.run.smithi193.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.0 MB/s | 5.1 MB 00:02 2024-05-14T16:49:46.994 INFO:teuthology.orchestra.run.smithi193.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-05-14T16:49:47.078 INFO:teuthology.orchestra.run.smithi193.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-05-14T16:49:47.120 INFO:teuthology.orchestra.run.smithi193.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-14T16:49:47.212 INFO:teuthology.orchestra.run.smithi193.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-05-14T16:49:47.246 INFO:teuthology.orchestra.run.smithi193.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 407 kB/s | 51 kB 00:00 2024-05-14T16:49:47.338 INFO:teuthology.orchestra.run.smithi193.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-05-14T16:49:47.426 INFO:teuthology.orchestra.run.smithi150.stdout:(102/156): uuid-1.6.2-55.el9.x86_64.rpm 56 kB/s | 58 kB 00:01 2024-05-14T16:49:47.555 INFO:teuthology.orchestra.run.smithi193.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 191 kB/s | 42 kB 00:00 2024-05-14T16:49:47.672 INFO:teuthology.orchestra.run.smithi193.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 381 kB/s | 44 kB 00:00 2024-05-14T16:49:47.735 INFO:teuthology.orchestra.run.smithi150.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 996 kB/s | 863 kB 00:00 2024-05-14T16:49:47.797 INFO:teuthology.orchestra.run.smithi193.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-05-14T16:49:47.914 INFO:teuthology.orchestra.run.smithi193.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-05-14T16:49:48.022 INFO:teuthology.orchestra.run.smithi193.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-05-14T16:49:48.139 INFO:teuthology.orchestra.run.smithi193.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-14T16:49:48.248 INFO:teuthology.orchestra.run.smithi193.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-05-14T16:49:48.365 INFO:teuthology.orchestra.run.smithi193.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 803 kB/s | 94 kB 00:00 2024-05-14T16:49:48.466 INFO:teuthology.orchestra.run.smithi193.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-05-14T16:49:48.557 INFO:teuthology.orchestra.run.smithi193.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 489 kB/s | 45 kB 00:00 2024-05-14T16:49:48.668 INFO:teuthology.orchestra.run.smithi150.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 331 kB/s | 309 kB 00:00 2024-05-14T16:49:48.727 INFO:teuthology.orchestra.run.smithi150.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 421 kB/s | 548 kB 00:01 2024-05-14T16:49:48.769 INFO:teuthology.orchestra.run.smithi150.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-14T16:49:48.878 INFO:teuthology.orchestra.run.smithi150.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-05-14T16:49:48.986 INFO:teuthology.orchestra.run.smithi150.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-05-14T16:49:49.124 INFO:teuthology.orchestra.run.smithi193.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-05-14T16:49:49.170 INFO:teuthology.orchestra.run.smithi150.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 368 kB/s | 67 kB 00:00 2024-05-14T16:49:49.445 INFO:teuthology.orchestra.run.smithi150.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 550 kB/s | 151 kB 00:00 2024-05-14T16:49:49.691 INFO:teuthology.orchestra.run.smithi193.stdout:(102/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.0 MB/s | 6.0 MB 00:06 2024-05-14T16:49:50.018 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-14T16:49:50.150 INFO:teuthology.orchestra.run.smithi193.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 842 kB/s | 863 kB 00:01 2024-05-14T16:49:50.358 INFO:teuthology.orchestra.run.smithi193.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 822 kB/s | 548 kB 00:00 2024-05-14T16:49:50.450 INFO:teuthology.orchestra.run.smithi193.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-05-14T16:49:50.520 INFO:teuthology.orchestra.run.smithi150.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 780 kB/s | 839 kB 00:01 2024-05-14T16:49:50.717 INFO:teuthology.orchestra.run.smithi193.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-05-14T16:49:50.817 INFO:teuthology.orchestra.run.smithi193.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-14T16:49:50.901 INFO:teuthology.orchestra.run.smithi193.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-14T16:49:50.926 INFO:teuthology.orchestra.run.smithi193.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-05-14T16:49:51.018 INFO:teuthology.orchestra.run.smithi193.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-05-14T16:49:51.052 INFO:teuthology.orchestra.run.smithi193.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-14T16:49:51.144 INFO:teuthology.orchestra.run.smithi193.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-05-14T16:49:51.244 INFO:teuthology.orchestra.run.smithi193.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-05-14T16:49:51.269 INFO:teuthology.orchestra.run.smithi193.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-05-14T16:49:51.321 INFO:teuthology.orchestra.run.smithi150.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 685 kB/s | 548 kB 00:00 2024-05-14T16:49:51.345 INFO:teuthology.orchestra.run.smithi193.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-05-14T16:49:51.370 INFO:teuthology.orchestra.run.smithi193.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-14T16:49:51.454 INFO:teuthology.orchestra.run.smithi193.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-14T16:49:51.454 INFO:teuthology.orchestra.run.smithi150.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 326 kB/s | 43 kB 00:00 2024-05-14T16:49:51.504 INFO:teuthology.orchestra.run.smithi193.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-05-14T16:49:51.554 INFO:teuthology.orchestra.run.smithi193.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-14T16:49:51.579 INFO:teuthology.orchestra.run.smithi150.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2024-05-14T16:49:51.604 INFO:teuthology.orchestra.run.smithi193.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-05-14T16:49:51.654 INFO:teuthology.orchestra.run.smithi193.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-14T16:49:51.755 INFO:teuthology.orchestra.run.smithi193.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-14T16:49:51.855 INFO:teuthology.orchestra.run.smithi193.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-14T16:49:51.897 INFO:teuthology.orchestra.run.smithi193.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-05-14T16:49:51.964 INFO:teuthology.orchestra.run.smithi193.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-05-14T16:49:51.997 INFO:teuthology.orchestra.run.smithi193.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-14T16:49:52.013 INFO:teuthology.orchestra.run.smithi150.stdout:(115/156): ceph-test-19.0.0-3687.g9b638668.el9. 4.5 MB/s | 49 MB 00:11 2024-05-14T16:49:52.038 INFO:teuthology.orchestra.run.smithi150.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 33 kB/s | 15 kB 00:00 2024-05-14T16:49:52.064 INFO:teuthology.orchestra.run.smithi193.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-14T16:49:52.098 INFO:teuthology.orchestra.run.smithi193.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-14T16:49:52.207 INFO:teuthology.orchestra.run.smithi193.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-14T16:49:52.232 INFO:teuthology.orchestra.run.smithi193.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-05-14T16:49:52.307 INFO:teuthology.orchestra.run.smithi193.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 788 kB/s | 79 kB 00:00 2024-05-14T16:49:52.319 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:49:52.320 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:49:52.331 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:49:52.331 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:49:52.332 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:49:52.333 INFO:teuthology.orchestra.run.smithi193.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-05-14T16:49:52.333 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:49:52.335 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:49:52.337 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:49:52.338 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:49:52.398 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-05-14T16:49:52.425 INFO:teuthology.orchestra.run.smithi193.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-14T16:49:52.427 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-05-14T16:49:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-05-14T16:49:52.427 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-05-14T16:49:52.427 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:49:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:49:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:49:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:49:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:49:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:49:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:49:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:49:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:49:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:49:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:49:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:49:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:49:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout:Install 148 Packages 2024-05-14T16:49:52.441 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 8 Packages 2024-05-14T16:49:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:49:52.442 INFO:teuthology.orchestra.run.smithi193.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-14T16:49:52.443 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 226 M 2024-05-14T16:49:52.443 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-05-14T16:49:52.525 INFO:teuthology.orchestra.run.smithi193.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-05-14T16:49:52.547 INFO:teuthology.orchestra.run.smithi150.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-05-14T16:49:52.550 INFO:teuthology.orchestra.run.smithi193.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-05-14T16:49:52.732 INFO:teuthology.orchestra.run.smithi150.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 240 kB/s | 172 kB 00:00 2024-05-14T16:49:52.810 INFO:teuthology.orchestra.run.smithi193.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 108 kB/s | 31 kB 00:00 2024-05-14T16:49:52.835 INFO:teuthology.orchestra.run.smithi193.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 665 kB/s | 188 kB 00:00 2024-05-14T16:49:52.839 INFO:teuthology.orchestra.run.smithi150.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-05-14T16:49:52.910 INFO:teuthology.orchestra.run.smithi193.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-05-14T16:49:52.935 INFO:teuthology.orchestra.run.smithi193.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-05-14T16:49:53.011 INFO:teuthology.orchestra.run.smithi193.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-05-14T16:49:53.015 INFO:teuthology.orchestra.run.smithi150.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 649 kB/s | 303 kB 00:00 2024-05-14T16:49:53.044 INFO:teuthology.orchestra.run.smithi193.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-14T16:49:53.111 INFO:teuthology.orchestra.run.smithi193.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-05-14T16:49:53.120 INFO:teuthology.orchestra.run.smithi099.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-05-14T16:49:53.123 INFO:teuthology.orchestra.run.smithi150.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 725 kB/s | 206 kB 00:00 2024-05-14T16:49:53.153 INFO:teuthology.orchestra.run.smithi193.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-14T16:49:53.212 INFO:teuthology.orchestra.run.smithi193.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-05-14T16:49:53.262 INFO:teuthology.orchestra.run.smithi193.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-14T16:49:53.354 INFO:teuthology.orchestra.run.smithi099.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 3.7 MB/s | 891 kB 00:00 2024-05-14T16:49:53.354 INFO:teuthology.orchestra.run.smithi193.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-14T16:49:53.399 INFO:teuthology.orchestra.run.smithi150.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 525 kB/s | 144 kB 00:00 2024-05-14T16:49:53.412 INFO:teuthology.orchestra.run.smithi099.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 2.4 MB/s | 145 kB 00:00 2024-05-14T16:49:53.499 INFO:teuthology.orchestra.run.smithi150.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-14T16:49:53.599 INFO:teuthology.orchestra.run.smithi150.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-05-14T16:49:53.699 INFO:teuthology.orchestra.run.smithi150.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-14T16:49:53.796 INFO:teuthology.orchestra.run.smithi193.stdout:(148/156): librados2-19.0.0-3687.g9b638668.el9. 6.4 MB/s | 3.4 MB 00:00 2024-05-14T16:49:53.800 INFO:teuthology.orchestra.run.smithi150.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-14T16:49:53.887 INFO:teuthology.orchestra.run.smithi099.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 5.0 MB/s | 2.4 MB 00:00 2024-05-14T16:49:53.908 INFO:teuthology.orchestra.run.smithi150.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-05-14T16:49:53.954 INFO:teuthology.orchestra.run.smithi193.stdout:(149/156): librbd1-19.0.0-3687.g9b638668.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-05-14T16:49:54.071 INFO:teuthology.orchestra.run.smithi193.stdout:(150/156): NetworkManager-1.47.90-1.el9.x86_64. 8.4 MB/s | 2.3 MB 00:00 2024-05-14T16:49:54.104 INFO:teuthology.orchestra.run.smithi099.stdout:(5/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-05-14T16:49:54.130 INFO:teuthology.orchestra.run.smithi193.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 714 kB/s | 41 kB 00:00 2024-05-14T16:49:54.188 INFO:teuthology.orchestra.run.smithi193.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-14T16:49:54.230 INFO:teuthology.orchestra.run.smithi193.stdout:(153/156): NetworkManager-libnm-1.47.90-1.el9.x 6.5 MB/s | 1.8 MB 00:00 2024-05-14T16:49:54.264 INFO:teuthology.orchestra.run.smithi193.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-05-14T16:49:54.292 INFO:teuthology.orchestra.run.smithi150.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 812 kB/s | 4.4 MB 00:05 2024-05-14T16:49:54.347 INFO:teuthology.orchestra.run.smithi193.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-05-14T16:49:54.392 INFO:teuthology.orchestra.run.smithi150.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-05-14T16:49:54.404 INFO:teuthology.orchestra.run.smithi099.stdout:(6/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-05-14T16:49:54.576 INFO:teuthology.orchestra.run.smithi150.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2024-05-14T16:49:54.663 INFO:teuthology.orchestra.run.smithi099.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-05-14T16:49:54.759 INFO:teuthology.orchestra.run.smithi150.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 315 kB/s | 58 kB 00:00 2024-05-14T16:49:55.096 INFO:teuthology.orchestra.run.smithi099.stdout:(8/156): ceph-common-19.0.0-3687.g9b638668.el9. 9.8 MB/s | 21 MB 00:02 2024-05-14T16:49:55.147 INFO:teuthology.orchestra.run.smithi099.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 504 kB/s | 25 kB 00:00 2024-05-14T16:49:55.185 INFO:teuthology.orchestra.run.smithi150.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 641 kB/s | 272 kB 00:00 2024-05-14T16:49:55.285 INFO:teuthology.orchestra.run.smithi150.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 759 kB/s | 1.0 MB 00:01 2024-05-14T16:49:55.310 INFO:teuthology.orchestra.run.smithi150.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-05-14T16:49:55.419 INFO:teuthology.orchestra.run.smithi150.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-05-14T16:49:55.469 INFO:teuthology.orchestra.run.smithi150.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 489 kB/s | 90 kB 00:00 2024-05-14T16:49:55.519 INFO:teuthology.orchestra.run.smithi150.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-05-14T16:49:55.662 INFO:teuthology.orchestra.run.smithi150.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 788 kB/s | 2.0 MB 00:02 2024-05-14T16:49:55.704 INFO:teuthology.orchestra.run.smithi150.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 321 kB/s | 59 kB 00:00 2024-05-14T16:49:55.779 INFO:teuthology.orchestra.run.smithi150.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 609 kB/s | 188 kB 00:00 2024-05-14T16:49:55.804 INFO:teuthology.orchestra.run.smithi150.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 251 kB/s | 36 kB 00:00 2024-05-14T16:49:55.829 INFO:teuthology.orchestra.run.smithi150.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-05-14T16:49:55.830 INFO:teuthology.orchestra.run.smithi099.stdout:(10/156): ceph-radosgw-19.0.0-3687.g9b638668.el 9.1 MB/s | 11 MB 00:01 2024-05-14T16:49:55.897 INFO:teuthology.orchestra.run.smithi099.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 3.3 MB/s | 225 kB 00:00 2024-05-14T16:49:55.972 INFO:teuthology.orchestra.run.smithi099.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 361 kB/s | 27 kB 00:00 2024-05-14T16:49:56.013 INFO:teuthology.orchestra.run.smithi150.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 430 kB/s | 90 kB 00:00 2024-05-14T16:49:56.031 INFO:teuthology.orchestra.run.smithi099.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 543 kB/s | 32 kB 00:00 2024-05-14T16:49:56.122 INFO:teuthology.orchestra.run.smithi150.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-05-14T16:49:56.156 INFO:teuthology.orchestra.run.smithi099.stdout:(14/156): libcephfs2-19.0.0-3687.g9b638668.el9. 5.8 MB/s | 748 kB 00:00 2024-05-14T16:49:56.163 INFO:teuthology.orchestra.run.smithi150.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 597 kB/s | 230 kB 00:00 2024-05-14T16:49:56.232 INFO:teuthology.orchestra.run.smithi099.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 2.1 MB/s | 163 kB 00:00 2024-05-14T16:49:56.299 INFO:teuthology.orchestra.run.smithi099.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 1.9 MB/s | 127 kB 00:00 2024-05-14T16:49:56.374 INFO:teuthology.orchestra.run.smithi099.stdout:(17/156): libradosstriper1-19.0.0-3687.g9b63866 6.5 MB/s | 501 kB 00:00 2024-05-14T16:49:56.430 INFO:teuthology.orchestra.run.smithi150.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 621 kB/s | 192 kB 00:00 2024-05-14T16:49:56.497 INFO:teuthology.orchestra.run.smithi150.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 639 kB/s | 427 kB 00:00 2024-05-14T16:49:57.124 INFO:teuthology.orchestra.run.smithi099.stdout:(18/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 6.2 MB/s | 17 MB 00:02 2024-05-14T16:49:57.189 INFO:teuthology.orchestra.run.smithi150.stdout:(148/156): librbd1-19.0.0-3687.g9b638668.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-05-14T16:49:57.191 INFO:teuthology.orchestra.run.smithi099.stdout:(19/156): librgw2-19.0.0-3687.g9b638668.el9.x86 6.6 MB/s | 5.4 MB 00:00 2024-05-14T16:49:57.268 INFO:teuthology.orchestra.run.smithi099.stdout:(20/156): python3-ceph-argparse-19.0.0-3687.g9b 315 kB/s | 45 kB 00:00 2024-05-14T16:49:57.300 INFO:teuthology.orchestra.run.smithi099.stdout:(21/156): python3-ceph-common-19.0.0-3687.g9b63 1.2 MB/s | 136 kB 00:00 2024-05-14T16:49:57.331 INFO:teuthology.orchestra.run.smithi150.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-05-14T16:49:57.375 INFO:teuthology.orchestra.run.smithi099.stdout:(22/156): python3-cephfs-19.0.0-3687.g9b638668. 1.5 MB/s | 161 kB 00:00 2024-05-14T16:49:57.402 INFO:teuthology.orchestra.run.smithi099.stdout:(23/156): python3-rados-19.0.0-3687.g9b638668.e 3.1 MB/s | 321 kB 00:00 2024-05-14T16:49:57.459 INFO:teuthology.orchestra.run.smithi099.stdout:(24/156): python3-rbd-19.0.0-3687.g9b638668.el9 5.0 MB/s | 299 kB 00:00 2024-05-14T16:49:57.493 INFO:teuthology.orchestra.run.smithi099.stdout:(25/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.1 MB/s | 100 kB 00:00 2024-05-14T16:49:57.527 INFO:teuthology.orchestra.run.smithi099.stdout:(26/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-14T16:49:57.610 INFO:teuthology.orchestra.run.smithi099.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 2.0 MB/s | 170 kB 00:00 2024-05-14T16:49:57.686 INFO:teuthology.orchestra.run.smithi099.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 455 kB/s | 34 kB 00:00 2024-05-14T16:49:57.794 INFO:teuthology.orchestra.run.smithi099.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.3 MB/s | 143 kB 00:00 2024-05-14T16:49:57.982 INFO:teuthology.orchestra.run.smithi150.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 2.8 MB/s | 1.8 MB 00:00 2024-05-14T16:49:58.040 INFO:teuthology.orchestra.run.smithi150.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 715 kB/s | 41 kB 00:00 2024-05-14T16:49:58.091 INFO:teuthology.orchestra.run.smithi150.stdout:(152/156): librados2-19.0.0-3687.g9b638668.el9. 2.0 MB/s | 3.4 MB 00:01 2024-05-14T16:49:58.158 INFO:teuthology.orchestra.run.smithi150.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 2.1 MB/s | 248 kB 00:00 2024-05-14T16:49:58.261 INFO:teuthology.orchestra.run.smithi150.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-14T16:49:58.273 INFO:teuthology.orchestra.run.smithi099.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 7.5 MB/s | 3.5 MB 00:00 2024-05-14T16:49:58.320 INFO:teuthology.orchestra.run.smithi099.stdout:(31/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 3.7 MB/s | 3.1 MB 00:00 2024-05-14T16:49:58.383 INFO:teuthology.orchestra.run.smithi150.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-14T16:49:58.583 INFO:teuthology.orchestra.run.smithi150.stdout:(156/156): thrift-0.15.0-2.el9.x86_64.rpm 669 kB/s | 1.6 MB 00:02 2024-05-14T16:49:58.592 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:49:58.592 INFO:teuthology.orchestra.run.smithi150.stdout:Total 11 MB/s | 226 MB 00:20 2024-05-14T16:49:58.738 INFO:teuthology.orchestra.run.smithi099.stdout:(32/156): ceph-test-19.0.0-3687.g9b638668.el9.x 14 MB/s | 49 MB 00:03 2024-05-14T16:49:58.762 INFO:teuthology.orchestra.run.smithi099.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3687.g9b 621 kB/s | 274 kB 00:00 2024-05-14T16:49:58.779 INFO:teuthology.orchestra.run.smithi099.stdout:(34/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 1.2 MB/s | 49 kB 00:00 2024-05-14T16:49:58.813 INFO:teuthology.orchestra.run.smithi099.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3687.g9 328 kB/s | 16 kB 00:00 2024-05-14T16:49:58.838 INFO:teuthology.orchestra.run.smithi099.stdout:(36/156): ceph-volume-19.0.0-3687.g9b638668.el9 4.6 MB/s | 273 kB 00:00 2024-05-14T16:49:58.888 INFO:teuthology.orchestra.run.smithi099.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 366 kB/s | 18 kB 00:00 2024-05-14T16:49:58.997 INFO:teuthology.orchestra.run.smithi099.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-05-14T16:49:59.031 INFO:teuthology.orchestra.run.smithi099.stdout:(39/156): cephadm-19.0.0-3687.g9b638668.el9.noa 3.4 MB/s | 763 kB 00:00 2024-05-14T16:49:59.056 INFO:teuthology.orchestra.run.smithi099.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 480 kB/s | 80 kB 00:00 2024-05-14T16:49:59.115 INFO:teuthology.orchestra.run.smithi099.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-14T16:49:59.198 INFO:teuthology.orchestra.run.smithi099.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-05-14T16:49:59.232 INFO:teuthology.orchestra.run.smithi099.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-05-14T16:49:59.257 INFO:teuthology.orchestra.run.smithi099.stdout:(44/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-05-14T16:49:59.299 INFO:teuthology.orchestra.run.smithi099.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-14T16:49:59.324 INFO:teuthology.orchestra.run.smithi099.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-14T16:49:59.350 INFO:teuthology.orchestra.run.smithi099.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-14T16:49:59.375 INFO:teuthology.orchestra.run.smithi099.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-14T16:49:59.459 INFO:teuthology.orchestra.run.smithi099.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-14T16:49:59.526 INFO:teuthology.orchestra.run.smithi099.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-05-14T16:49:59.551 INFO:teuthology.orchestra.run.smithi099.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-05-14T16:49:59.610 INFO:teuthology.orchestra.run.smithi099.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-05-14T16:49:59.668 INFO:teuthology.orchestra.run.smithi099.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-05-14T16:49:59.735 INFO:teuthology.orchestra.run.smithi099.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-05-14T16:49:59.742 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-14T16:49:59.761 INFO:teuthology.orchestra.run.smithi099.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-05-14T16:49:59.786 INFO:teuthology.orchestra.run.smithi099.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-14T16:49:59.861 INFO:teuthology.orchestra.run.smithi099.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 605 kB/s | 45 kB 00:00 2024-05-14T16:49:59.895 INFO:teuthology.orchestra.run.smithi099.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-14T16:49:59.896 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-14T16:49:59.896 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-14T16:50:00.079 INFO:teuthology.orchestra.run.smithi099.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-05-14T16:50:00.129 INFO:teuthology.orchestra.run.smithi099.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-05-14T16:50:00.187 INFO:teuthology.orchestra.run.smithi099.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-05-14T16:50:00.313 INFO:teuthology.orchestra.run.smithi099.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.4 MB/s | 662 kB 00:00 2024-05-14T16:50:00.338 INFO:teuthology.orchestra.run.smithi099.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-14T16:50:00.397 INFO:teuthology.orchestra.run.smithi099.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-05-14T16:50:00.455 INFO:teuthology.orchestra.run.smithi099.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-14T16:50:00.539 INFO:teuthology.orchestra.run.smithi099.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-05-14T16:50:00.814 INFO:teuthology.orchestra.run.smithi099.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-05-14T16:50:01.206 INFO:teuthology.orchestra.run.smithi099.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 5.9 MB/s | 4.7 MB 00:00 2024-05-14T16:50:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-14T16:50:01.373 INFO:teuthology.orchestra.run.smithi099.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 1.6 MB/s | 911 kB 00:00 2024-05-14T16:50:01.432 INFO:teuthology.orchestra.run.smithi099.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-05-14T16:50:01.515 INFO:teuthology.orchestra.run.smithi099.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-14T16:50:01.674 INFO:teuthology.orchestra.run.smithi099.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-05-14T16:50:01.728 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-14T16:50:01.728 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-14T16:50:01.799 INFO:teuthology.orchestra.run.smithi099.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-14T16:50:01.933 INFO:teuthology.orchestra.run.smithi099.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 9.1 MB/s | 5.8 MB 00:00 2024-05-14T16:50:01.967 INFO:teuthology.orchestra.run.smithi099.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-05-14T16:50:01.984 INFO:teuthology.orchestra.run.smithi099.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-14T16:50:02.034 INFO:teuthology.orchestra.run.smithi099.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 285 kB/s | 19 kB 00:00 2024-05-14T16:50:02.084 INFO:teuthology.orchestra.run.smithi099.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-05-14T16:50:02.110 INFO:teuthology.orchestra.run.smithi099.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-05-14T16:50:02.252 INFO:teuthology.orchestra.run.smithi099.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-05-14T16:50:02.391 INFO:teuthology.orchestra.run.smithi099.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.8 MB/s | 6.0 MB 00:00 2024-05-14T16:50:02.477 INFO:teuthology.orchestra.run.smithi099.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 902 kB/s | 77 kB 00:00 2024-05-14T16:50:02.511 INFO:teuthology.orchestra.run.smithi099.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 858 kB/s | 222 kB 00:00 2024-05-14T16:50:02.578 INFO:teuthology.orchestra.run.smithi099.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-05-14T16:50:02.653 INFO:teuthology.orchestra.run.smithi099.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-05-14T16:50:02.728 INFO:teuthology.orchestra.run.smithi099.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 5.1 MB 00:00 2024-05-14T16:50:02.754 INFO:teuthology.orchestra.run.smithi099.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 905 kB/s | 159 kB 00:00 2024-05-14T16:50:02.829 INFO:teuthology.orchestra.run.smithi099.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 716 kB/s | 54 kB 00:00 2024-05-14T16:50:02.846 INFO:teuthology.orchestra.run.smithi099.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-05-14T16:50:02.905 INFO:teuthology.orchestra.run.smithi099.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-05-14T16:50:02.980 INFO:teuthology.orchestra.run.smithi099.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 854 kB/s | 279 kB 00:00 2024-05-14T16:50:03.064 INFO:teuthology.orchestra.run.smithi099.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-14T16:50:03.114 INFO:teuthology.orchestra.run.smithi099.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-05-14T16:50:03.131 INFO:teuthology.orchestra.run.smithi099.stdout:(95/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 195 kB/s | 44 kB 00:00 2024-05-14T16:50:03.181 INFO:teuthology.orchestra.run.smithi099.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-05-14T16:50:03.390 INFO:teuthology.orchestra.run.smithi099.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-05-14T16:50:03.473 INFO:teuthology.orchestra.run.smithi099.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 198 kB/s | 58 kB 00:00 2024-05-14T16:50:03.549 INFO:teuthology.orchestra.run.smithi099.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 591 kB/s | 94 kB 00:00 2024-05-14T16:50:03.566 INFO:teuthology.orchestra.run.smithi099.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-05-14T16:50:03.616 INFO:teuthology.orchestra.run.smithi099.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-05-14T16:50:03.683 INFO:teuthology.orchestra.run.smithi099.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 334 kB/s | 45 kB 00:00 2024-05-14T16:50:03.750 INFO:teuthology.orchestra.run.smithi099.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-05-14T16:50:04.501 INFO:teuthology.orchestra.run.smithi099.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.7 MB/s | 16 MB 00:01 2024-05-14T16:50:04.609 INFO:teuthology.orchestra.run.smithi099.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 592 kB/s | 548 kB 00:00 2024-05-14T16:50:04.643 INFO:teuthology.orchestra.run.smithi099.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 346 kB/s | 309 kB 00:00 2024-05-14T16:50:04.818 INFO:teuthology.orchestra.run.smithi099.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 149 kB/s | 26 kB 00:00 2024-05-14T16:50:04.860 INFO:teuthology.orchestra.run.smithi099.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-05-14T16:50:04.894 INFO:teuthology.orchestra.run.smithi099.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-05-14T16:50:05.044 INFO:teuthology.orchestra.run.smithi099.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-05-14T16:50:05.111 INFO:teuthology.orchestra.run.smithi099.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-05-14T16:50:05.136 INFO:teuthology.orchestra.run.smithi099.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 624 kB/s | 151 kB 00:00 2024-05-14T16:50:05.262 INFO:teuthology.orchestra.run.smithi099.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-05-14T16:50:05.321 INFO:teuthology.orchestra.run.smithi099.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-05-14T16:50:05.354 INFO:teuthology.orchestra.run.smithi099.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 839 kB 00:00 2024-05-14T16:50:05.421 INFO:teuthology.orchestra.run.smithi099.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 203 kB/s | 32 kB 00:00 2024-05-14T16:50:05.471 INFO:teuthology.orchestra.run.smithi099.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 99 kB/s | 15 kB 00:00 2024-05-14T16:50:05.530 INFO:teuthology.orchestra.run.smithi099.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 977 kB/s | 172 kB 00:00 2024-05-14T16:50:05.580 INFO:teuthology.orchestra.run.smithi099.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-05-14T16:50:05.664 INFO:teuthology.orchestra.run.smithi099.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-05-14T16:50:05.688 INFO:teuthology.orchestra.run.smithi193.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 902 kB/s | 16 MB 00:18 2024-05-14T16:50:05.691 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:50:05.692 INFO:teuthology.orchestra.run.smithi193.stdout:Total 6.8 MB/s | 226 MB 00:33 2024-05-14T16:50:05.705 INFO:teuthology.orchestra.run.smithi099.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 201 kB/s | 35 kB 00:00 2024-05-14T16:50:05.731 INFO:teuthology.orchestra.run.smithi099.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-05-14T16:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-14T16:50:05.832 INFO:teuthology.orchestra.run.smithi099.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-05-14T16:50:05.882 INFO:teuthology.orchestra.run.smithi099.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-14T16:50:05.907 INFO:teuthology.orchestra.run.smithi099.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-05-14T16:50:05.949 INFO:teuthology.orchestra.run.smithi099.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-05-14T16:50:05.974 INFO:teuthology.orchestra.run.smithi099.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 210 kB/s | 19 kB 00:00 2024-05-14T16:50:05.999 INFO:teuthology.orchestra.run.smithi099.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-05-14T16:50:06.058 INFO:teuthology.orchestra.run.smithi099.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-14T16:50:06.083 INFO:teuthology.orchestra.run.smithi099.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 425 kB/s | 46 kB 00:00 2024-05-14T16:50:06.117 INFO:teuthology.orchestra.run.smithi099.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 670 kB/s | 79 kB 00:00 2024-05-14T16:50:06.143 INFO:teuthology.orchestra.run.smithi099.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 686 kB/s | 58 kB 00:00 2024-05-14T16:50:06.168 INFO:teuthology.orchestra.run.smithi099.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-14T16:50:06.193 INFO:teuthology.orchestra.run.smithi099.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-14T16:50:06.218 INFO:teuthology.orchestra.run.smithi099.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-14T16:50:06.244 INFO:teuthology.orchestra.run.smithi099.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-14T16:50:06.269 INFO:teuthology.orchestra.run.smithi099.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-05-14T16:50:06.294 INFO:teuthology.orchestra.run.smithi099.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-14T16:50:06.320 INFO:teuthology.orchestra.run.smithi099.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-14T16:50:06.345 INFO:teuthology.orchestra.run.smithi099.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-05-14T16:50:06.370 INFO:teuthology.orchestra.run.smithi099.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-14T16:50:06.404 INFO:teuthology.orchestra.run.smithi099.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-14T16:50:06.429 INFO:teuthology.orchestra.run.smithi099.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-14T16:50:06.479 INFO:teuthology.orchestra.run.smithi099.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-14T16:50:06.496 INFO:teuthology.orchestra.run.smithi099.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-05-14T16:50:06.588 INFO:teuthology.orchestra.run.smithi099.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-05-14T16:50:06.780 INFO:teuthology.orchestra.run.smithi099.stdout:(148/156): librados2-19.0.0-3687.g9b638668.el9. 12 MB/s | 3.4 MB 00:00 2024-05-14T16:50:06.834 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-14T16:50:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-05-14T16:50:06.987 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-14T16:50:06.987 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-14T16:50:07.047 INFO:teuthology.orchestra.run.smithi099.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-14T16:50:07.089 INFO:teuthology.orchestra.run.smithi099.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 999 kB/s | 41 kB 00:00 2024-05-14T16:50:07.140 INFO:teuthology.orchestra.run.smithi099.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-14T16:50:07.207 INFO:teuthology.orchestra.run.smithi099.stdout:(153/156): thrift-0.15.0-2.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-05-14T16:50:07.240 INFO:teuthology.orchestra.run.smithi099.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-05-14T16:50:07.374 INFO:teuthology.orchestra.run.smithi099.stdout:(155/156): librbd1-19.0.0-3687.g9b638668.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-05-14T16:50:07.432 INFO:teuthology.orchestra.run.smithi099.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-14T16:50:07.439 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:50:07.439 INFO:teuthology.orchestra.run.smithi099.stdout:Total 15 MB/s | 226 MB 00:15 2024-05-14T16:50:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-14T16:50:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:50:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:50:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:50:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:50:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:50:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:50:08.652 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-05-14T16:50:08.728 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:50:08.782 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:08.798 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-14T16:50:08.798 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-14T16:50:08.805 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-05-14T16:50:08.805 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-05-14T16:50:08.907 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:08.935 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:09.027 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:09.056 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:50:09.220 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:50:09.271 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:50:09.299 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:50:09.454 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:50:09.476 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:50:09.494 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:50:09.693 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:50:09.833 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:50:09.990 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:50:10.034 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:50:10.128 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:10.185 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:10.381 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:10.492 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:50:10.522 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:50:10.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:50:10.622 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:50:10.656 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:50:10.701 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:50:10.741 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:50:10.769 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:50:10.838 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:50:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:50:10.922 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:50:10.975 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-05-14T16:50:10.975 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-05-14T16:50:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:50:11.077 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:50:11.120 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:50:11.149 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:50:11.218 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:50:11.272 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:50:11.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:50:11.480 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:50:11.534 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:50:12.055 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:50:12.136 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:50:12.165 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:50:12.208 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:50:12.246 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:50:12.288 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:50:12.441 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:50:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:50:12.582 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:50:12.647 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:50:12.720 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:50:12.751 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:50:12.787 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:50:12.834 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:50:12.870 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:50:12.913 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:50:12.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:50:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:50:13.144 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:50:13.174 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:50:13.215 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:50:13.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:50:13.369 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:50:13.406 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:50:13.432 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:50:13.456 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:50:13.530 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:50:13.658 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:50:13.713 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:50:13.741 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:50:13.776 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:50:13.817 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:50:13.849 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:50:13.887 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:50:13.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:50:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:50:13.997 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:50:14.030 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:50:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:50:14.170 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:50:14.201 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:50:14.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:50:14.337 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:50:14.385 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:50:14.993 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-14T16:50:15.067 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:50:15.105 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:50:15.113 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:50:15.166 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:50:15.167 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:50:15.200 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:50:15.201 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:50:15.227 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:50:15.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:50:15.298 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:50:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:50:15.436 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:50:15.477 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:50:15.596 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:50:15.655 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:50:15.683 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:50:15.719 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:15.761 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:50:15.850 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:15.879 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:15.962 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:15.991 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:50:16.139 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:50:16.205 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:50:16.233 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:16.336 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:16.359 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:50:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:50:16.421 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:50:16.435 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:50:16.457 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:50:16.522 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:50:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:50:16.780 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:50:16.922 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:50:16.967 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:50:17.067 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:17.071 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-05-14T16:50:17.135 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:17.162 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:50:17.204 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:50:17.263 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:50:17.295 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:50:17.321 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:50:17.331 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:17.372 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:50:17.440 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:50:17.472 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:50:17.507 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:50:17.579 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:50:17.615 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:50:17.667 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:50:17.699 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:50:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:50:17.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:50:17.748 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:50:17.797 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:50:17.813 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:17.820 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:50:17.848 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:50:17.848 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:50:17.880 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:50:17.881 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:50:17.915 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:50:17.935 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:17.964 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:17.992 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:50:18.035 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:50:18.065 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:18.078 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:50:18.094 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:50:18.107 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:50:18.175 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:50:18.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:50:18.221 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:50:18.228 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:50:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:50:18.295 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:50:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:50:18.318 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:50:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:18.349 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:50:18.386 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:50:18.427 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:50:18.448 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:18.473 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:50:18.479 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:50:18.542 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:50:18.563 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:50:18.588 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:50:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:50:18.871 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:50:18.904 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:50:18.926 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:50:18.952 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:50:18.976 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:50:18.999 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:50:19.077 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:50:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:50:19.107 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:50:19.138 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:50:19.150 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:50:19.189 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:50:19.229 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:50:19.245 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:19.323 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:19.392 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:50:19.449 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:50:19.533 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:50:19.539 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:50:19.596 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:50:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:50:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:50:19.668 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:50:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:50:19.700 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:50:19.737 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:50:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:50:19.784 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:50:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:50:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:50:19.844 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:50:19.874 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:50:19.889 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:50:19.916 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:50:19.931 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:50:19.984 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:50:20.038 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:50:20.066 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:50:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:50:20.104 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:50:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:50:20.134 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:50:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:50:20.181 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:50:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:50:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:50:20.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:50:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:50:20.309 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:50:20.349 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:50:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:50:20.374 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:50:20.399 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:50:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:50:20.482 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:50:20.486 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:50:20.600 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:50:20.615 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:50:20.632 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:50:20.659 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:50:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:50:20.694 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:50:20.733 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:50:20.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:50:20.794 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:50:20.829 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:50:20.862 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:50:20.904 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:50:20.939 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:50:21.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:50:21.069 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:50:21.108 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:50:21.155 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:50:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:50:21.252 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:50:21.276 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:50:21.296 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:21.301 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:50:21.305 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:50:21.345 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:50:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:50:21.429 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:50:21.586 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:50:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:50:21.738 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:50:21.812 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:50:21.885 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:50:21.886 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:21.916 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:50:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:50:21.952 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:50:21.999 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:50:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:50:22.016 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:50:22.016 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:50:22.016 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:22.020 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:50:22.035 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:50:22.042 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:50:22.073 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:50:22.077 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:50:22.107 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:50:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:50:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:50:22.256 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:50:22.265 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:50:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:50:22.360 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:50:22.392 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:50:22.541 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:50:22.569 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:50:22.607 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:50:22.608 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:50:22.684 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:50:22.704 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:50:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:50:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:50:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:50:22.941 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:50:23.195 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:50:23.324 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:50:23.399 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:50:23.430 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:50:23.430 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:50:23.458 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:50:23.492 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:50:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:50:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:50:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:50:23.652 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:50:23.686 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:50:23.727 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:50:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:50:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:50:23.886 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:50:23.926 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:50:23.973 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:50:24.070 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:50:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:50:24.656 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:50:24.736 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:50:24.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:50:24.797 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:50:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:50:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:50:24.925 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:50:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:50:25.083 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:50:25.110 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:50:25.152 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:50:25.170 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:50:25.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:50:25.226 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:50:25.256 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:50:25.293 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:50:25.387 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:50:25.416 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:50:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:50:25.772 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:50:25.804 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:50:25.884 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:50:25.906 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:50:26.158 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:50:26.273 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:50:27.527 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:50:27.604 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:50:27.641 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:50:27.674 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:50:27.987 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:50:28.054 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:50:28.144 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:50:28.167 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:50:28.204 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:50:28.213 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:28.252 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:28.682 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:50:28.715 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:50:28.770 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:50:28.789 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:28.800 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:50:28.839 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:50:28.930 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:50:28.931 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:50:28.931 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:50:28.931 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:28.957 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:50:31.124 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:50:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:50:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:50:31.850 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:50:31.850 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:50:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-05-14T16:50:33.525 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-05-14T16:50:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:33.699 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:50:33.716 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:50:33.716 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:33.716 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:50:33.716 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:35.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:50:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:50:35.220 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:35.220 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:50:35.220 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:50:35.220 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:50:35.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:35.453 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:50:35.475 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:50:35.476 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:35.476 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:50:35.476 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:50:35.476 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:50:35.476 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:50:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:50:35.951 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:35.951 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:50:35.951 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:50:35.951 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:50:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:36.066 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:50:36.123 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:50:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:50:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:50:36.502 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:50:36.589 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:50:36.729 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:50:36.766 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:50:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:50:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:50:37.227 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:50:37.257 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:50:37.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:50:37.351 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:50:37.380 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:50:38.417 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:50:38.466 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:50:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:50:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:50:39.568 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:50:39.727 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:50:39.760 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:50:39.783 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:50:39.783 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:39.784 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:50:39.784 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:50:39.784 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:50:39.784 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:39.827 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:50:39.838 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:50:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:50:39.984 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-05-14T16:50:40.584 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:40.755 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:50:40.771 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:50:40.771 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:40.771 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:50:40.771 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:40.909 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:50:40.946 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:50:40.969 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:50:40.969 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:40.969 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:50:40.969 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:50:40.970 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:50:40.970 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:41.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:50:41.036 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:50:41.036 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:41.036 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:50:41.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:41.081 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:50:41.104 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:50:41.104 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:41.104 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:50:41.104 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:50:41.104 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:50:41.104 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:41.415 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:50:41.437 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:50:41.437 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:41.437 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:50:41.438 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:50:41.438 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:50:41.438 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:42.259 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:50:42.284 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:50:42.284 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:42.284 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:50:42.284 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:50:42.284 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:50:42.284 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:42.509 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:50:42.531 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:50:42.531 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:42.531 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:50:42.531 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:50:42.531 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:50:42.531 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:42.985 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:50:43.007 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:50:43.007 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:43.007 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:50:43.007 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:50:43.007 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:50:43.007 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:43.139 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:50:43.187 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:50:43.242 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-05-14T16:50:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:43.426 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:50:43.491 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:50:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:50:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:50:43.559 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:43.559 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:50:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:43.568 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:50:43.656 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:50:43.786 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:50:43.825 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:50:43.865 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:50:43.893 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:50:44.330 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:50:44.359 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:50:44.391 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:50:44.453 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:50:44.482 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:50:45.046 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:50:45.072 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:50:45.072 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:45.072 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:50:45.072 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:50:45.072 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:50:45.073 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:50:45.320 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:50:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:50:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:50:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:50:45.320 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:45.520 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:50:45.550 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:50:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:50:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:50:45.787 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:45.787 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:50:45.787 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:50:45.787 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:50:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:50:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:50:46.014 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:50:46.068 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:50:46.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:50:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:50:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:50:46.201 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:50:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:50:46.248 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:50:46.326 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:50:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:50:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:50:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:50:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:50:46.491 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:50:46.512 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:50:46.537 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:50:46.542 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:50:46.553 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:50:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:50:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:50:46.652 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:50:46.658 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:50:46.814 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:50:46.836 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:50:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:50:46.860 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:46.860 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:50:46.860 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:50:46.860 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:50:46.860 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:46.921 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:50:46.932 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:50:47.029 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:50:47.077 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:50:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:50:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:50:47.160 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:50:47.202 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:47.221 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:50:47.263 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:50:48.001 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:50:48.040 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:50:48.054 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:50:48.063 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:50:48.063 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:48.063 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:50:48.063 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:50:48.063 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:50:48.063 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:48.107 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:50:48.129 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:50:48.129 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:48.129 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:50:48.129 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:48.147 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:50:48.175 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:50:48.189 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:50:48.197 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:50:48.197 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:48.197 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:50:48.197 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:50:48.197 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:50:48.197 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:48.269 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:50:48.298 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:50:48.301 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:50:48.327 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:50:48.327 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:50:48.339 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:50:48.395 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:50:48.419 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:50:48.419 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:50:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:50:48.509 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:50:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:50:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:50:48.531 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:48.531 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:50:48.531 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:50:48.531 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:50:48.531 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:50:48.705 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:48.705 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:50:48.789 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:50:48.853 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:50:49.305 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:50:49.340 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:50:49.450 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:50:49.611 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:50:49.642 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:50:49.666 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:50:49.667 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:49.667 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:50:49.667 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:50:49.667 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:50:49.667 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:49.718 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:50:49.730 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:50:49.827 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:50:49.884 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:50:50.812 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:50:50.837 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:50:50.861 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:50:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:50:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:50:50.936 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:50.936 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:50:50.936 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:50.989 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:50:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:50:51.012 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:51.012 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:50:51.012 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:50:51.012 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:50:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:51.314 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:50:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:50:51.338 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:50:51.338 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:50:51.338 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:50:51.338 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:50:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:50:53.137 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:50:53.187 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:50:53.236 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:50:53.269 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:50:53.296 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:50:53.479 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:50:53.525 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:50:53.556 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:50:53.586 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:50:53.632 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:50:54.227 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:54.264 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:54.283 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:50:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:50:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:50:54.396 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:50:54.397 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:50:54.398 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:50:54.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:50:54.400 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:50:54.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:50:54.402 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:50:54.403 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:50:54.404 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:50:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:50:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:50:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:50:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:50:55.105 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:50:55.186 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:50:55.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:50:55.318 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:50:55.355 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:50:55.382 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:50:55.382 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:50:55.468 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:50:55.492 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:50:55.492 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:50:55.584 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:50:55.609 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:50:55.614 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:55.614 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:55.803 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:55.803 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:50:55.898 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:50:55.952 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:50:55.958 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:50:56.000 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:50:56.042 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:50:56.075 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:50:56.102 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:50:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:50:56.346 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:50:56.376 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:50:56.406 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:50:56.452 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:50:56.986 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:50:56.987 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.988 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.989 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:50:56.990 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.991 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:50:56.992 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:50:56.993 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:50:56.994 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:50:56.995 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:50:56.996 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:50:56.997 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:50:56.998 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:50:56.999 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:50:56.999 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-14T16:50:56.999 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-14T16:50:57.071 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:57.116 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:57.137 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:50:57.422 DEBUG:teuthology.parallel:result is None 2024-05-14T16:50:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:50:58.070 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:50:58.117 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:50:58.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:50:58.239 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:50:58.265 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:50:58.265 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:50:58.358 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:50:58.400 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:50:58.400 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:50:58.484 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:50:58.509 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:50:58.514 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:58.515 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:58.702 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:50:58.702 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:50:58.787 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:50:58.854 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:51:01.509 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:51:01.509 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:51:01.509 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:51:01.509 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:51:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:51:01.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:51:01.523 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:51:01.523 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:51:01.523 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:51:01.524 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:51:01.525 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:51:01.526 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:51:01.527 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:51:01.528 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:51:01.529 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:51:01.530 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:51:01.531 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:51:01.532 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:51:01.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:51:01.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:51:01.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:51:01.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:51:01.533 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:51:04.362 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:51:04.363 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:51:04.364 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:51:04.364 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:51:04.364 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:51:04.364 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:51:04.364 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:51:04.376 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:51:04.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:51:04.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:51:04.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:51:04.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:51:04.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:51:04.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:51:04.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:51:04.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:51:04.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:51:04.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:51:04.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:51:04.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:51:04.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:51:04.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:04.405 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.406 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:51:04.408 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:51:04.409 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:51:04.410 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:51:04.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:51:04.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:51:04.415 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:51:04.416 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:51:04.417 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:51:04.417 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:51:04.417 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-14T16:51:04.835 DEBUG:teuthology.parallel:result is None 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:51:07.218 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.219 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:51:07.220 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.221 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:51:07.222 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:51:07.223 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:51:07.224 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:51:07.225 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:51:07.226 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:51:07.227 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:51:07.228 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:51:07.229 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-05-14T16:51:07.637 DEBUG:teuthology.parallel:result is None 2024-05-14T16:51:07.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:51:07.929 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:51:07.952 INFO:teuthology.orchestra.run.smithi099.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:51:07.953 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:51:07.953 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:51:07.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:51:08.280 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:51:08.305 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:51:08.306 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:51:08.306 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:51:08.307 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:51:08.449 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:51:08.473 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:51:08.474 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:51:08.474 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:51:08.475 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-14T16:51:08.476 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:51:08.476 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:51:08.508 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:51:08.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:51:08.539 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:51:08.539 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:51:08.568 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-14T16:51:08.568 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:51:08.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:51:08.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:51:08.663 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:51:08.663 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:51:08.690 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:51:08.756 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:51:08.756 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:51:08.782 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:51:08.848 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-14T16:51:08.848 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:51:08.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:51:08.875 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:51:08.942 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:51:08.942 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:51:08.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:51:09.036 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:51:09.036 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:51:09.062 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:51:09.129 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-14T16:51:09.129 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:51:09.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:51:09.157 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:51:09.223 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:51:09.224 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:51:09.247 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:51:09.311 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:51:09.312 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:51:09.338 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:51:09.404 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-14T16:51:09.514 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-14T16:51:09.515 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:51:09.515 INFO:tasks.cephadm:Cluster fsid is 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:51:09.515 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-14T16:51:09.516 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.99', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.193'} 2024-05-14T16:51:09.516 INFO:tasks.cephadm:First mon is mon.a on smithi099 2024-05-14T16:51:09.516 INFO:tasks.cephadm:First mgr is x 2024-05-14T16:51:09.516 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-14T16:51:09.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-05-14T16:51:09.544 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-05-14T16:51:09.571 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-05-14T16:51:09.598 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-14T16:51:09.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:51:09.881 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79371/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3687-g9b638668', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3687.g9b638668'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-14 15:14:57.349842', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-14T16:51:10.167 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce, sha1 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:51:10.168 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-14T16:51:10.168 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-14T16:51:10.169 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:10.630 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:51 /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:10.630 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.047 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:51 /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.048 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.498 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:51 /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.498 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.518 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.536 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:51:11.571 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b on all hosts... 2024-05-14T16:51:11.571 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:51:11.573 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:51:11.578 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:51:12.436 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:51:12.529 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:51:12.574 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:51:45.348 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T16:51:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:51:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:51:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-05-14T16:51:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:51:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T16:51:45.349 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T16:51:45.557 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T16:51:45.557 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:51:45.557 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:51:45.557 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-05-14T16:51:45.557 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:51:45.557 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-05-14T16:51:45.558 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T16:51:46.594 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-14T16:51:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:51:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:51:46.594 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-05-14T16:51:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:51:46.595 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-14T16:51:46.595 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-14T16:51:46.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-05-14T16:51:46.652 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-05-14T16:51:46.678 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-05-14T16:51:46.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-05-14T16:51:46.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-05-14T16:51:46.763 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-05-14T16:51:46.791 INFO:tasks.cephadm:Writing seed config... 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-14T16:51:46.792 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-14T16:51:46.793 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-14T16:51:46.794 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-14T16:51:46.795 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-14T16:51:46.796 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:51:46.796 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-14T16:51:46.814 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2a006d80-1212-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-14T16:51:46.814 DEBUG:teuthology.orchestra.run.smithi099:mon.a> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service 2024-05-14T16:51:46.857 DEBUG:teuthology.orchestra.run.smithi099:mgr.x> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x.service 2024-05-14T16:51:46.901 INFO:tasks.cephadm:Bootstrapping... 2024-05-14T16:51:46.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b -v bootstrap --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.99 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:51:47.097 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:51:47.097 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', '-v', 'bootstrap', '--fsid', '2a006d80-1212-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.99', '--skip-admin-label'] 2024-05-14T16:51:47.097 INFO:teuthology.orchestra.run.smithi099.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-14T16:51:47.098 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying podman|docker is present... 2024-05-14T16:51:47.124 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 5.0.2 2024-05-14T16:51:47.124 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying lvm2 is present... 2024-05-14T16:51:47.125 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying time synchronization is in place... 2024-05-14T16:51:47.132 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T16:51:47.132 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T16:51:47.138 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T16:51:47.138 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-05-14T16:51:47.145 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-05-14T16:51:47.151 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-05-14T16:51:47.151 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-05-14T16:51:47.152 INFO:teuthology.orchestra.run.smithi099.stdout:Repeating the final host check... 2024-05-14T16:51:47.175 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 5.0.2 2024-05-14T16:51:47.175 INFO:teuthology.orchestra.run.smithi099.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-14T16:51:47.175 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl is present 2024-05-14T16:51:47.175 INFO:teuthology.orchestra.run.smithi099.stdout:lvcreate is present 2024-05-14T16:51:47.181 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T16:51:47.181 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T16:51:47.187 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T16:51:47.187 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-05-14T16:51:47.194 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Host looks OK 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Cluster fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Acquiring lock 140366357336272 on /run/cephadm/2a006d80-1212-11ef-bc98-c7b262605968.lock 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Lock 140366357336272 acquired on /run/cephadm/2a006d80-1212-11ef-bc98-c7b262605968.lock 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 3300 ... 2024-05-14T16:51:47.201 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 6789 ... 2024-05-14T16:51:47.202 INFO:teuthology.orchestra.run.smithi099.stdout:Base mon IP(s) is [172.21.15.99:3300, 172.21.15.99:6789], mon addrv is [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2024-05-14T16:51:47.205 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.99 metric 100 2024-05-14T16:51:47.205 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.99 metric 100 2024-05-14T16:51:47.207 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-14T16:51:47.207 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-14T16:51:47.208 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1711sec hoplimit 64 pref medium 2024-05-14T16:51:47.210 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-14T16:51:47.210 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-14T16:51:47.210 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T16:51:47.210 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-14T16:51:47.210 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-05-14T16:51:47.210 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T16:51:47.211 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-05-14T16:51:47.211 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-05-14T16:51:47.211 INFO:teuthology.orchestra.run.smithi099.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-14T16:51:47.211 INFO:teuthology.orchestra.run.smithi099.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-14T16:51:47.212 INFO:teuthology.orchestra.run.smithi099.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Getting image source signatures 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:959e8422c821254139eba7e305bc0e5ccc9179a9c74388bbc27d9cf0e33e5891 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying config sha256:c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-14T16:51:48.834 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-14T16:51:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:ceph: stdout ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:51:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph version: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:51:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:Extracting ceph user uid/gid from container image... 2024-05-14T16:51:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:stat: stdout 167 167 2024-05-14T16:51:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial keys... 2024-05-14T16:52:02.612 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQCslkNmdsjjCRAAkgJi0kzFEjhSxz0a1qO83A== 2024-05-14T16:52:04.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQC0lkNmevf5AxAAV8jy1Renr1A0jMwy9B/Zdg== 2024-05-14T16:52:05.515 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQC1lkNm6uxFARAARVLo7iqxmn89y7u3drk9aA== 2024-05-14T16:52:05.515 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial monmap... 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool for a [v2:172.21.15.99:3300,v1:172.21.15.99:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = quincy 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: set fsid to 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T16:52:06.480 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:52:06.481 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mon... 2024-05-14T16:52:07.411 INFO:teuthology.orchestra.run.smithi099.stdout:create mon.a on 2024-05-14T16:52:07.614 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-14T16:52:07.752 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-14T16:52:07.942 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2a006d80-1212-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-2a006d80-1212-11ef-bc98-c7b262605968.target. 2024-05-14T16:52:07.943 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2a006d80-1212-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-2a006d80-1212-11ef-bc98-c7b262605968.target. 2024-05-14T16:52:08.180 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a 2024-05-14T16:52:08.180 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-14T16:52:08.348 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a006d80-1212-11ef-bc98-c7b262605968.target.wants/ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2a006d80-1212-11ef-bc98-c7b262605968@.service. 2024-05-14T16:52:09.128 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:52:09.128 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-05-14T16:52:09.135 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:52:09.135 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-05-14T16:52:09.135 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-05-14T16:52:09.135 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T16:52:09.136 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon to start... 2024-05-14T16:52:09.136 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon... 2024-05-14T16:52:10.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:10 smithi099 ceph-mon[31481]: from='client.? 172.21.15.99:0/4056328104' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-14T16:52:10.625 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout cluster: 2024-05-14T16:52:10.625 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout id: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:10.625 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout services: 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.90577s) 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout data: 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pgs: 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.937+0000 7f7b8bf7a700 1 Processor -- start 2024-05-14T16:52:10.626 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.937+0000 7f7b8bf7a700 1 -- start start 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.937+0000 7f7b8bf7a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84109940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.937+0000 7f7b8bf7a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84109f10 con 0x7f7b84109520 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b89d16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b89d16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35150/0 (socket says 172.21.15.99:35150) 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b89d16700 1 -- 172.21.15.99:0/1556863574 learned_addr learned my addr 172.21.15.99:0/1556863574 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b89d16700 1 -- 172.21.15.99:0/1556863574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b84108180 con 0x7f7b84109520 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b89d16700 1 --2- 172.21.15.99:0/1556863574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84109940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009e80 tx=0x7f7b7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1d44a338995995 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b88d14700 1 -- 172.21.15.99:0/1556863574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b7c017070 con 0x7f7b84109520 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b88d14700 1 -- 172.21.15.99:0/1556863574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c003a20 con 0x7f7b84109520 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.938+0000 7f7b88d14700 1 -- 172.21.15.99:0/1556863574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b7c003d20 con 0x7f7b84109520 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/1556863574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 msgr2=0x7f7b84109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 --2- 172.21.15.99:0/1556863574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84109940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009e80 tx=0x7f7b7c00b330 comp rx=0 tx=0).stop 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/1556863574 shutdown_connections 2024-05-14T16:52:10.627 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 --2- 172.21.15.99:0/1556863574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84109940 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/1556863574 >> 172.21.15.99:0/1556863574 conn(0x7f7b84102cb0 msgr2=0x7f7b841050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/1556863574 shutdown_connections 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.939+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/1556863574 wait complete. 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b8bf7a700 1 Processor -- start 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b8bf7a700 1 -- start start 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b8bf7a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84107b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b8bf7a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84108650 con 0x7f7b84109520 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b89d16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b89d16700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84107b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35160/0 (socket says 172.21.15.99:35160) 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b89d16700 1 -- 172.21.15.99:0/4056328104 learned_addr learned my addr 172.21.15.99:0/4056328104 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b89d16700 1 -- 172.21.15.99:0/4056328104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8419f510 con 0x7f7b84109520 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b89d16700 1 --2- 172.21.15.99:0/4056328104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84107b50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c004000 tx=0x7f7b7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b7c022020 con 0x7f7b84109520 2024-05-14T16:52:10.628 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.940+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c018040 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.941+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b7c017070 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.941+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8419f6e0 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.941+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8419fb60 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.941+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b7c025020 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.941+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b7c00f050 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.941+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b84064ba0 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:09.944+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7b7c02c030 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.013+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7b8402e870 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.013+0000 7f7b7affd700 1 -- 172.21.15.99:0/4056328104 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f7b7c01e480 con 0x7f7b84109520 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 msgr2=0x7f7b84107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 --2- 172.21.15.99:0/4056328104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84107b50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c004000 tx=0x7f7b7c004ad0 comp rx=0 tx=0).stop 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 shutdown_connections 2024-05-14T16:52:10.629 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 --2- 172.21.15.99:0/4056328104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84109520 0x7f7b84107b50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:10.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 >> 172.21.15.99:0/4056328104 conn(0x7f7b84102cb0 msgr2=0x7f7b841922e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:10.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 shutdown_connections 2024-05-14T16:52:10.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:10.014+0000 7f7b8bf7a700 1 -- 172.21.15.99:0/4056328104 wait complete. 2024-05-14T16:52:10.630 INFO:teuthology.orchestra.run.smithi099.stdout:mon is available 2024-05-14T16:52:10.630 INFO:teuthology.orchestra.run.smithi099.stdout:Assimilating anything we can from ceph.conf... 2024-05-14T16:52:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:11 smithi099 ceph-mon[31481]: from='client.? 172.21.15.99:0/1030692028' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T16:52:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:11 smithi099 ceph-mon[31481]: from='client.? 172.21.15.99:0/1030692028' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-14T16:52:11.614 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.008+0000 7fab2abfa700 1 Processor -- start 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.008+0000 7fab2abfa700 1 -- start start 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.008+0000 7fab2abfa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab241094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.008+0000 7fab2abfa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab24109a20 con 0x7fab241090c0 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab28996700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab241094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab28996700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab241094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35174/0 (socket says 172.21.15.99:35174) 2024-05-14T16:52:11.616 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab28996700 1 -- 172.21.15.99:0/2369754188 learned_addr learned my addr 172.21.15.99:0/2369754188 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab28996700 1 -- 172.21.15.99:0/2369754188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab24109ba0 con 0x7fab241090c0 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab28996700 1 --2- 172.21.15.99:0/2369754188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab241094e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fab18009e80 tx=0x7fab1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeced95f6109749b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab237fe700 1 -- 172.21.15.99:0/2369754188 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab18017070 con 0x7fab241090c0 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab237fe700 1 -- 172.21.15.99:0/2369754188 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab18003a20 con 0x7fab241090c0 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.009+0000 7fab237fe700 1 -- 172.21.15.99:0/2369754188 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab18003d20 con 0x7fab241090c0 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 -- 172.21.15.99:0/2369754188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 msgr2=0x7fab241094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 --2- 172.21.15.99:0/2369754188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab241094e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fab18009e80 tx=0x7fab1800b330 comp rx=0 tx=0).stop 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 -- 172.21.15.99:0/2369754188 shutdown_connections 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 --2- 172.21.15.99:0/2369754188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab241094e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 -- 172.21.15.99:0/2369754188 >> 172.21.15.99:0/2369754188 conn(0x7fab24104720 msgr2=0x7fab24106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 -- 172.21.15.99:0/2369754188 shutdown_connections 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 -- 172.21.15.99:0/2369754188 wait complete. 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 Processor -- start 2024-05-14T16:52:11.617 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.010+0000 7fab2abfa700 1 -- start start 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab2abfa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab2419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab2abfa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab2410a520 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab28996700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab2419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab28996700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab2419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35176/0 (socket says 172.21.15.99:35176) 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab28996700 1 -- 172.21.15.99:0/1030692028 learned_addr learned my addr 172.21.15.99:0/1030692028 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab28996700 1 -- 172.21.15.99:0/1030692028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2419ccf0 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab28996700 1 --2- 172.21.15.99:0/1030692028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab2419c7b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fab24109df0 tx=0x7fab18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab18022020 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab18018040 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2419cf80 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.011+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab18017070 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.012+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2419fc70 con 0x7fab241090c0 2024-05-14T16:52:11.618 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.012+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab18025020 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.012+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab1800f050 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.012+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab24064ba0 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.015+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fab1802c030 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.081+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fab2402e720 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.085+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fab1801e480 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.085+0000 7fab21ffb700 1 -- 172.21.15.99:0/1030692028 <== mon.0 v2:172.21.15.99:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fab18003a20 con 0x7fab241090c0 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 msgr2=0x7fab2419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 --2- 172.21.15.99:0/1030692028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab2419c7b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fab24109df0 tx=0x7fab18004ad0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 shutdown_connections 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 --2- 172.21.15.99:0/1030692028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241090c0 0x7fab2419c7b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 >> 172.21.15.99:0/1030692028 conn(0x7fab24104720 msgr2=0x7fab24193410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 shutdown_connections 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:11.086+0000 7fab2abfa700 1 -- 172.21.15.99:0/1030692028 wait complete. 2024-05-14T16:52:11.619 INFO:teuthology.orchestra.run.smithi099.stdout:Generating new minimal ceph.conf... 2024-05-14T16:52:12.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:12 smithi099 ceph-mon[31481]: from='client.? 172.21.15.99:0/711222204' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:12.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.031+0000 7fac81bbb700 1 Processor -- start 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.031+0000 7fac81bbb700 1 -- start start 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.031+0000 7fac81bbb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.031+0000 7fac81bbb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c108870 con 0x7fac7c107ef0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35178/0 (socket says 172.21.15.99:35178) 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7b7fe700 1 -- 172.21.15.99:0/2971835482 learned_addr learned my addr 172.21.15.99:0/2971835482 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7b7fe700 1 -- 172.21.15.99:0/2971835482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c1090d0 con 0x7fac7c107ef0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7b7fe700 1 --2- 172.21.15.99:0/2971835482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c108330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098a0 tx=0x7fac6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cce7d591ddef50 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7affd700 1 -- 172.21.15.99:0/2971835482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c004160 con 0x7fac7c107ef0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7affd700 1 -- 172.21.15.99:0/2971835482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac6c004300 con 0x7fac7c107ef0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.032+0000 7fac7affd700 1 -- 172.21.15.99:0/2971835482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c011af0 con 0x7fac7c107ef0 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 -- 172.21.15.99:0/2971835482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 msgr2=0x7fac7c108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:12.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 --2- 172.21.15.99:0/2971835482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c108330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098a0 tx=0x7fac6c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 -- 172.21.15.99:0/2971835482 shutdown_connections 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 --2- 172.21.15.99:0/2971835482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c108330 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 -- 172.21.15.99:0/2971835482 >> 172.21.15.99:0/2971835482 conn(0x7fac7c1034c0 msgr2=0x7fac7c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 -- 172.21.15.99:0/2971835482 shutdown_connections 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.033+0000 7fac81bbb700 1 -- 172.21.15.99:0/2971835482 wait complete. 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac81bbb700 1 Processor -- start 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac81bbb700 1 -- start start 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac81bbb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac81bbb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c109f60 con 0x7fac7c107ef0 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac7b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35180/0 (socket says 172.21.15.99:35180) 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac7b7fe700 1 -- 172.21.15.99:0/711222204 learned_addr learned my addr 172.21.15.99:0/711222204 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:12.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac7b7fe700 1 -- 172.21.15.99:0/711222204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c19c4a0 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac7b7fe700 1 --2- 172.21.15.99:0/711222204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c19bf60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0117b0 tx=0x7fac6c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c003af0 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac6c020070 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c019630 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.034+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c19c730 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.035+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c19f420 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.035+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac6c019c80 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.035+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac6c00b720 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.035+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7c064ba0 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.038+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fac6c0042a0 con 0x7fac7c107ef0 2024-05-14T16:52:12.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.104+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fac7c19f960 con 0x7fac7c107ef0 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.105+0000 7fac797fa700 1 -- 172.21.15.99:0/711222204 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fac6c01a9c0 con 0x7fac7c107ef0 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.105+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 msgr2=0x7fac7c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.105+0000 7fac81bbb700 1 --2- 172.21.15.99:0/711222204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c19bf60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0117b0 tx=0x7fac6c011b70 comp rx=0 tx=0).stop 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.106+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 shutdown_connections 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.106+0000 7fac81bbb700 1 --2- 172.21.15.99:0/711222204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c107ef0 0x7fac7c19bf60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.106+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 >> 172.21.15.99:0/711222204 conn(0x7fac7c1034c0 msgr2=0x7fac7c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.106+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 shutdown_connections 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:12.106+0000 7fac81bbb700 1 -- 172.21.15.99:0/711222204 wait complete. 2024-05-14T16:52:12.656 INFO:teuthology.orchestra.run.smithi099.stdout:Restarting the monitor... 2024-05-14T16:52:12.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:12 smithi099 systemd[1]: Stopping Ceph mon.a for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:52:13.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a[31475]: 2024-05-14T16:52:12.988+0000 7fa7e48da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T16:52:13.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a[31475]: 2024-05-14T16:52:12.988+0000 7fa7e48da700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-14T16:52:13.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:13 smithi099 podman[31729]: 2024-05-14 16:52:13.235109397 +0000 UTC m=+0.421032932 container died e5aa441b34f2b26163c4cbea3c464e290b05c62d83a5e5c986423cebe18d3bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:52:13.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:13 smithi099 podman[31729]: 2024-05-14 16:52:13.732620813 +0000 UTC m=+0.918544348 container cleanup e5aa441b34f2b26163c4cbea3c464e290b05c62d83a5e5c986423cebe18d3bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-14T16:52:13.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:13 smithi099 bash[31729]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a 2024-05-14T16:52:14.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:13 smithi099 podman[31744]: 2024-05-14 16:52:13.991987251 +0000 UTC m=+0.749614624 container remove e5aa441b34f2b26163c4cbea3c464e290b05c62d83a5e5c986423cebe18d3bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-14T16:52:14.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T16:52:14.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 systemd[1]: Stopped Ceph mon.a for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:52:14.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 systemd[1]: Starting Ceph mon.a for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:52:14.594 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 podman[31832]: 2024-05-14 16:52:14.276245054 +0000 UTC m=+0.017920933 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:52:14.594 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 podman[31832]: 2024-05-14 16:52:14.3760887 +0000 UTC m=+0.117764568 container create 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True) 2024-05-14T16:52:14.594 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 podman[31832]: 2024-05-14 16:52:14.595046248 +0000 UTC m=+0.336722116 container init 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:52:14.673 INFO:teuthology.orchestra.run.smithi099.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-14T16:52:14.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 podman[31832]: 2024-05-14 16:52:14.598032917 +0000 UTC m=+0.339708785 container start 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T16:52:14.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T16:52:14.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-14T16:52:14.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: pidfile_write: ignore empty --pid-file 2024-05-14T16:52:14.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: load: jerasure load: lrc 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: RocksDB version: 7.9.2 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Git sha 0 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: DB SUMMARY 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: DB Session ID: 8GQXOLI1F12Q4EW6YI4J 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: CURRENT file: CURRENT 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-14T16:52:14.873 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105438 ; 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.error_if_exists: 0 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.create_if_missing: 0 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.paranoid_checks: 1 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T16:52:14.874 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.env: 0x5604f33d0cc0 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.info_log: 0x5604f4115da0 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.statistics: (nil) 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.use_fsync: 0 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_log_file_size: 0 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T16:52:14.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.allow_fallocate: 1 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.use_direct_reads: 0 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T16:52:14.876 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.db_log_dir: 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.wal_dir: 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T16:52:14.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.write_buffer_manager: 0x5604f40e70e0 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T16:52:14.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.unordered_write: 0 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.row_cache: None 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.wal_filter: None 2024-05-14T16:52:14.879 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.two_write_queues: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.wal_compression: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.atomic_flush: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.log_readahead_size: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T16:52:14.880 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_background_jobs: 2 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_background_compactions: -1 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_subcompactions: 1 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_open_files: -1 2024-05-14T16:52:14.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_background_flushes: -1 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Compression algorithms supported: 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kZSTD supported: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kXpressCompression supported: 0 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kZlibCompression supported: 1 2024-05-14T16:52:14.882 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kSnappyCompression supported: 1 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kLZ4Compression supported: 1 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: kBZip2Compression supported: 0 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.merge_operator: 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_filter: None 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T16:52:14.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5604f4115f60) 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_type: 0 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_index_type: 0 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_shortening: 1 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: checksum: 4 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: no_block_cache: 0 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache: 0x5604f41191f0 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_options: 2024-05-14T16:52:14.884 INFO:journalctl@ceph.mon.a.smithi099.stdout: capacity : 536870912 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_shard_bits : 4 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: strict_capacity_limit : 0 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_compressed: (nil) 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: persistent_cache: (nil) 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size: 4096 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size_deviation: 10 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_restart_interval: 16 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_block_restart_interval: 1 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: metadata_block_size: 4096 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: partition_filters: 0 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: use_delta_encoding: 1 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: filter_policy: bloomfilter 2024-05-14T16:52:14.885 INFO:journalctl@ceph.mon.a.smithi099.stdout: whole_key_filtering: 1 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: verify_compression: 0 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: format_version: 5 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_index_compression: 1 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_align: 0 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: max_auto_readahead_size: 262144 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: prepopulate_block_cache: 0 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: initial_auto_readahead_size: 8192 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression: NoCompression 2024-05-14T16:52:14.886 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T16:52:14.887 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T16:52:14.887 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T16:52:14.887 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.num_levels: 7 2024-05-14T16:52:14.887 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T16:52:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T16:52:14.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T16:52:14.890 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.inplace_update_support: 0 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T16:52:14.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.bloom_locality: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.max_successive_merges: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.ttl: 2592000 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.enable_blob_files: false 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.min_blob_size: 0 2024-05-14T16:52:14.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T16:52:14.893 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 46c6c008-5533-41bd-9720-03c2ca1ac971 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715705534627493, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715705534628506, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100550, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98435, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90689, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715705534, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "46c6c008-5533-41bd-9720-03c2ca1ac971", "db_session_id": "8GQXOLI1F12Q4EW6YI4J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715705534628587, "job": 1, "event": "recovery_finished"} 2024-05-14T16:52:14.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5604f4236000 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: DB pointer 0x5604f4222000 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** DB Stats ** 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T16:52:14.895 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T16:52:14.896 INFO:journalctl@ceph.mon.a.smithi099.stdout: L0 2/0 100.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: Sum 2/0 100.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 152.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-05-14T16:52:14.897 INFO:journalctl@ceph.mon.a.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative compaction: 0.00 GB write, 27.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:52:14.898 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval compaction: 0.00 GB write, 27.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache BinnedLRUCache@0x5604f41191f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 7e-06 secs_since: 0 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: starting mon.a rank 0 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???) e1 preinit fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).mds e1 new map 2024-05-14T16:52:14.899 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).mds e1 print_map 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout: e1 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout: legacy client fscid: -1 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout: No filesystems configured 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:52:14.900 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 bash[31832]: 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 systemd[1]: Started Ceph mon.a for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: monmap epoch 1 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: last_changed 2024-05-14T16:52:05.874052+0000 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: created 2024-05-14T16:52:05.874052+0000 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: min_mon_release 19 (squid) 2024-05-14T16:52:14.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: election_strategy: 1 2024-05-14T16:52:14.902 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: 0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-05-14T16:52:14.902 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: fsmap 2024-05-14T16:52:14.902 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T16:52:14.902 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:14 smithi099 ceph-mon[31851]: mgrmap e1: no daemons active 2024-05-14T16:52:15.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f322e0f1700 1 Processor -- start 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f322e0f1700 1 -- start start 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f322e0f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f3228109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f322e0f1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32281099b0 con 0x7f3228109050 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f32277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f3228109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f32277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f3228109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36448/0 (socket says 172.21.15.99:36448) 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.075+0000 7f32277fe700 1 -- 172.21.15.99:0/946236623 learned_addr learned my addr 172.21.15.99:0/946236623 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:15.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.076+0000 7f32277fe700 1 -- 172.21.15.99:0/946236623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3228109b30 con 0x7f3228109050 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.076+0000 7f32277fe700 1 --2- 172.21.15.99:0/946236623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f3228109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32180098a0 tx=0x7f3218009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfe3506769e06c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.076+0000 7f3226ffd700 1 -- 172.21.15.99:0/946236623 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218004160 con 0x7f3228109050 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.076+0000 7f3226ffd700 1 -- 172.21.15.99:0/946236623 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3218004300 con 0x7f3228109050 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.076+0000 7f3226ffd700 1 -- 172.21.15.99:0/946236623 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218011af0 con 0x7f3228109050 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 -- 172.21.15.99:0/946236623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 msgr2=0x7f3228109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 --2- 172.21.15.99:0/946236623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f3228109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32180098a0 tx=0x7f3218009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 -- 172.21.15.99:0/946236623 shutdown_connections 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 --2- 172.21.15.99:0/946236623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f3228109470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:15.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 -- 172.21.15.99:0/946236623 >> 172.21.15.99:0/946236623 conn(0x7f3228104670 msgr2=0x7f3228106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 -- 172.21.15.99:0/946236623 shutdown_connections 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 -- 172.21.15.99:0/946236623 wait complete. 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 Processor -- start 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.077+0000 7f322e0f1700 1 -- start start 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f322e0f1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f322819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f322e0f1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f322810b050 con 0x7f3228109050 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f322819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32277fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f322819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36458/0 (socket says 172.21.15.99:36458) 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32277fe700 1 -- 172.21.15.99:0/4044666368 learned_addr learned my addr 172.21.15.99:0/4044666368 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32277fe700 1 -- 172.21.15.99:0/4044666368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322819d520 con 0x7f3228109050 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32277fe700 1 --2- 172.21.15.99:0/4044666368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f322819cfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3218009fb0 tx=0x7f3218011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32180036a0 con 0x7f3228109050 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f321801a430 con 0x7f3228109050 2024-05-14T16:52:15.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322819d7b0 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218019550 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281a04a0 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.078+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3218019be0 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.079+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3218024730 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.079+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3228064ba0 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.081+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3218043ec0 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.147+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f32281a0670 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.149+0000 7f32257fa700 1 -- 172.21.15.99:0/4044666368 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3218032070 con 0x7f3228109050 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 msgr2=0x7f322819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 --2- 172.21.15.99:0/4044666368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f322819cfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3218009fb0 tx=0x7f3218011940 comp rx=0 tx=0).stop 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 shutdown_connections 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 --2- 172.21.15.99:0/4044666368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3228109050 0x7f322819cfe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 >> 172.21.15.99:0/4044666368 conn(0x7f3228104670 msgr2=0x7f3228105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 shutdown_connections 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:15.150+0000 7f322e0f1700 1 -- 172.21.15.99:0/4044666368 wait complete. 2024-05-14T16:52:15.663 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-14T16:52:15.674 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:52:15.675 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mgr... 2024-05-14T16:52:15.675 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-14T16:52:15.675 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-14T16:52:15.919 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x 2024-05-14T16:52:15.919 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-14T16:52:16.064 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a006d80-1212-11ef-bc98-c7b262605968.target.wants/ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2a006d80-1212-11ef-bc98-c7b262605968@.service. 2024-05-14T16:52:16.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:16 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4044666368' entity='client.admin' 2024-05-14T16:52:16.562 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:16 smithi099 podman[32067]: 2024-05-14 16:52:16.492860582 +0000 UTC m=+0.308652221 container start f9791659271ca4a669423acb1ea59707e91968265ca491f2798ef6bc4f6a8d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-14T16:52:16.563 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:16 smithi099 bash[32067]: f9791659271ca4a669423acb1ea59707e91968265ca491f2798ef6bc4f6a8d27 2024-05-14T16:52:16.578 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:52:16.579 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-05-14T16:52:16.588 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:52:16.588 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-05-14T16:52:16.588 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-05-14T16:52:16.588 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T16:52:16.598 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:52:16.598 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-05-14T16:52:16.607 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:52:16.607 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-05-14T16:52:16.607 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-05-14T16:52:16.607 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-14T16:52:16.607 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr to start... 2024-05-14T16:52:16.607 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr... 2024-05-14T16:52:16.834 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:16 smithi099 systemd[1]: Started Ceph mgr.x for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:52:16.834 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:16.655+0000 7fd4432d9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:52:17.130 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:16.849+0000 7fd4432d9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:52:17.130 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:16.955+0000 7fd4432d9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:52:17.444 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:17 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4104969710' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:52:17.445 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:17 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:17.283+0000 7fd4432d9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:52:17.445 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:17 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:17.371+0000 7fd4432d9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:52:17.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:17.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:52:17.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:52:17.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:52:17.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:52:17.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:17.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:17.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:52:09.109836+0000", 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.051+0000 7f3c37be1700 1 Processor -- start 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.051+0000 7f3c37be1700 1 -- start start 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.051+0000 7f3c37be1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c30108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.051+0000 7f3c37be1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30108890 con 0x7f3c30107f10 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3597d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c30108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3597d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c30108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36472/0 (socket says 172.21.15.99:36472) 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3597d700 1 -- 172.21.15.99:0/3740151122 learned_addr learned my addr 172.21.15.99:0/3740151122 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3597d700 1 -- 172.21.15.99:0/3740151122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c301090f0 con 0x7f3c30107f10 2024-05-14T16:52:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3597d700 1 --2- 172.21.15.99:0/3740151122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c30108350 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098a0 tx=0x7f3c2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d1cfbd0bf71f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3497b700 1 -- 172.21.15.99:0/3740151122 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c2c004160 con 0x7f3c30107f10 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3497b700 1 -- 172.21.15.99:0/3740151122 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c2c004300 con 0x7f3c30107f10 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.052+0000 7f3c3497b700 1 -- 172.21.15.99:0/3740151122 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c2c011af0 con 0x7f3c30107f10 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 -- 172.21.15.99:0/3740151122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 msgr2=0x7f3c30108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 --2- 172.21.15.99:0/3740151122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c30108350 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098a0 tx=0x7f3c2c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 -- 172.21.15.99:0/3740151122 shutdown_connections 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 --2- 172.21.15.99:0/3740151122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c30108350 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 -- 172.21.15.99:0/3740151122 >> 172.21.15.99:0/3740151122 conn(0x7f3c301034e0 msgr2=0x7f3c30105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 -- 172.21.15.99:0/3740151122 shutdown_connections 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.053+0000 7f3c37be1700 1 -- 172.21.15.99:0/3740151122 wait complete. 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c37be1700 1 Processor -- start 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c37be1700 1 -- start start 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c37be1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c300769e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c37be1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30109f80 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c3597d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c300769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c3597d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c300769e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36480/0 (socket says 172.21.15.99:36480) 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.054+0000 7f3c3597d700 1 -- 172.21.15.99:0/4104969710 learned_addr learned my addr 172.21.15.99:0/4104969710 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c3597d700 1 -- 172.21.15.99:0/4104969710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3019fd50 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c3597d700 1 --2- 172.21.15.99:0/4104969710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c300769e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009fb0 tx=0x7f3c2c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c2c0036a0 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c2c01a430 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3019ff20 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c2c019550 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.055+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c301a02c0 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.056+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c2c019b90 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.056+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c2c02dbd0 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.056+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c30064ba0 con 0x7f3c30107f10 2024-05-14T16:52:17.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.058+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3c2c043ec0 con 0x7f3c30107f10 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.130+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3c301a0600 con 0x7f3c30107f10 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.131+0000 7f3c26ffd700 1 -- 172.21.15.99:0/4104969710 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3c2c032020 con 0x7f3c30107f10 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 msgr2=0x7f3c300769e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 --2- 172.21.15.99:0/4104969710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c300769e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009fb0 tx=0x7f3c2c011940 comp rx=0 tx=0).stop 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 shutdown_connections 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 --2- 172.21.15.99:0/4104969710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c30107f10 0x7f3c300769e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 >> 172.21.15.99:0/4104969710 conn(0x7f3c301034e0 msgr2=0x7f3c301040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 shutdown_connections 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:17.132+0000 7f3c37be1700 1 -- 172.21.15.99:0/4104969710 wait complete. 2024-05-14T16:52:17.681 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (1/15)... 2024-05-14T16:52:18.925 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:18.587+0000 7fd4432d9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:52:19.046 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:18.755+0000 7fd4432d9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:52:19.046 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:18.924+0000 7fd4432d9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:52:19.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:19.211+0000 7fd4432d9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:52:19.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:19.305+0000 7fd4432d9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:52:19.794 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:19.522+0000 7fd4432d9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:52:19.795 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:19.720+0000 7fd4432d9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:52:20.354 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:20 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2491500893' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:52:20.683 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:20.354+0000 7fd4432d9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:52:20.683 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:20.581+0000 7fd4432d9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:52:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:52:20.689 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:52:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:20.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:52:09.109836+0000", 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.080+0000 7f32a7f3e700 1 Processor -- start 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.081+0000 7f32a7f3e700 1 -- start start 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.081+0000 7f32a7f3e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.081+0000 7f32a7f3e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0109ee0 con 0x7f32a0109560 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.081+0000 7f32a5cda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.081+0000 7f32a5cda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36484/0 (socket says 172.21.15.99:36484) 2024-05-14T16:52:20.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.081+0000 7f32a5cda700 1 -- 172.21.15.99:0/1377952627 learned_addr learned my addr 172.21.15.99:0/1377952627 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.082+0000 7f32a5cda700 1 -- 172.21.15.99:0/1377952627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a0107b60 con 0x7f32a0109560 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.082+0000 7f32a5cda700 1 --2- 172.21.15.99:0/1377952627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01099a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098a0 tx=0x7f329c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9d950c08d2ea88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.082+0000 7f32a4cd8700 1 -- 172.21.15.99:0/1377952627 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c004160 con 0x7f32a0109560 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.082+0000 7f32a4cd8700 1 -- 172.21.15.99:0/1377952627 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f329c004300 con 0x7f32a0109560 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.082+0000 7f32a4cd8700 1 -- 172.21.15.99:0/1377952627 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c011af0 con 0x7f32a0109560 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 -- 172.21.15.99:0/1377952627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 msgr2=0x7f32a01099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 --2- 172.21.15.99:0/1377952627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01099a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098a0 tx=0x7f329c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 -- 172.21.15.99:0/1377952627 shutdown_connections 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 --2- 172.21.15.99:0/1377952627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01099a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 -- 172.21.15.99:0/1377952627 >> 172.21.15.99:0/1377952627 conn(0x7f32a0103500 msgr2=0x7f32a0105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 -- 172.21.15.99:0/1377952627 shutdown_connections 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 -- 172.21.15.99:0/1377952627 wait complete. 2024-05-14T16:52:20.701 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.083+0000 7f32a7f3e700 1 Processor -- start 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a7f3e700 1 -- start start 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a7f3e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01930b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a7f3e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32a010c390 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a5cda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a5cda700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36500/0 (socket says 172.21.15.99:36500) 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a5cda700 1 -- 172.21.15.99:0/2491500893 learned_addr learned my addr 172.21.15.99:0/2491500893 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a5cda700 1 -- 172.21.15.99:0/2491500893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a01935f0 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a5cda700 1 --2- 172.21.15.99:0/2491500893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01930b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f329c003f40 tx=0x7f329c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c0038b0 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f329c024430 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a0191830 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.084+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c01a430 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.085+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a0191d10 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.085+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f329c01aa70 con 0x7f32a0109560 2024-05-14T16:52:20.702 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.085+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f329c0193c0 con 0x7f32a0109560 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.085+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a0064ba0 con 0x7f32a0109560 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.087+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f329c043ec0 con 0x7f32a0109560 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.161+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f32a0196cb0 con 0x7f32a0109560 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.162+0000 7f3296ffd700 1 -- 172.21.15.99:0/2491500893 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f329c032020 con 0x7f32a0109560 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 msgr2=0x7f32a01930b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 --2- 172.21.15.99:0/2491500893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01930b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f329c003f40 tx=0x7f329c011890 comp rx=0 tx=0).stop 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 shutdown_connections 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 --2- 172.21.15.99:0/2491500893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a0109560 0x7f32a01930b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 >> 172.21.15.99:0/2491500893 conn(0x7f32a0103500 msgr2=0x7f32a0104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 shutdown_connections 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:20.163+0000 7f32a7f3e700 1 -- 172.21.15.99:0/2491500893 wait complete. 2024-05-14T16:52:20.703 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (2/15)... 2024-05-14T16:52:21.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:21.006+0000 7fd4432d9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:52:21.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:21.101+0000 7fd4432d9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:52:21.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:21.470+0000 7fd4432d9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:52:22.158 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:21.826+0000 7fd4432d9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:52:22.158 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:21.918+0000 7fd4432d9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:52:22.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:22.158+0000 7fd4432d9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:52:22.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:22.292+0000 7fd4432d9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:52:22.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:22.387+0000 7fd4432d9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:52:23.010 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:22.716+0000 7fd4432d9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:52:23.010 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:22.809+0000 7fd4432d9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:52:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:23 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/872132478' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:52:23.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:23 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:23.439+0000 7fd4432d9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:52:23.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:23 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:23.658+0000 7fd4432d9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:52:23.785 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:52:23.786 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:52:23.787 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:52:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:52:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:52:09.109836+0000", 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa37b321700 1 Processor -- start 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa37b321700 1 -- start start 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa37b321700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa374108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa37b321700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa374108890 con 0x7fa374107f10 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa3790bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa374108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa3790bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa374108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36502/0 (socket says 172.21.15.99:36502) 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.112+0000 7fa3790bd700 1 -- 172.21.15.99:0/1601340150 learned_addr learned my addr 172.21.15.99:0/1601340150 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa3790bd700 1 -- 172.21.15.99:0/1601340150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3741090f0 con 0x7fa374107f10 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa3790bd700 1 --2- 172.21.15.99:0/1601340150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa374108350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa3700098a0 tx=0x7fa370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89a67893fa0c954 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa36bfff700 1 -- 172.21.15.99:0/1601340150 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa370004160 con 0x7fa374107f10 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa36bfff700 1 -- 172.21.15.99:0/1601340150 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa370004300 con 0x7fa374107f10 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa36bfff700 1 -- 172.21.15.99:0/1601340150 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa370011af0 con 0x7fa374107f10 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa37b321700 1 -- 172.21.15.99:0/1601340150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 msgr2=0x7fa374108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.113+0000 7fa37b321700 1 --2- 172.21.15.99:0/1601340150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa374108350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa3700098a0 tx=0x7fa370009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 -- 172.21.15.99:0/1601340150 shutdown_connections 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 --2- 172.21.15.99:0/1601340150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa374108350 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 -- 172.21.15.99:0/1601340150 >> 172.21.15.99:0/1601340150 conn(0x7fa3741034e0 msgr2=0x7fa374105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:23.798 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 -- 172.21.15.99:0/1601340150 shutdown_connections 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 -- 172.21.15.99:0/1601340150 wait complete. 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 Processor -- start 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.114+0000 7fa37b321700 1 -- start start 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa37b321700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa37419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa37b321700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa374109f80 con 0x7fa374107f10 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa3790bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa37419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa3790bd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa37419d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36518/0 (socket says 172.21.15.99:36518) 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa3790bd700 1 -- 172.21.15.99:0/872132478 learned_addr learned my addr 172.21.15.99:0/872132478 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa3790bd700 1 -- 172.21.15.99:0/872132478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37419d570 con 0x7fa374107f10 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa3790bd700 1 --2- 172.21.15.99:0/872132478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa37419d030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa3700117b0 tx=0x7fa370011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa370003af0 con 0x7fa374107f10 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37419d800 con 0x7fa374107f10 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa370020070 con 0x7fa374107f10 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa370019630 con 0x7fa374107f10 2024-05-14T16:52:23.799 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.115+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3741a04f0 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.116+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa370019c30 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.116+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa37000ba40 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.116+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa374064ba0 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.118+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa3700042c0 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.191+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa3741a0a30 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.192+0000 7fa36a7fc700 1 -- 172.21.15.99:0/872132478 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa37001a9c0 con 0x7fa374107f10 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 msgr2=0x7fa37419d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 --2- 172.21.15.99:0/872132478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa37419d030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa3700117b0 tx=0x7fa370011b70 comp rx=0 tx=0).stop 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 shutdown_connections 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 --2- 172.21.15.99:0/872132478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa374107f10 0x7fa37419d030 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 >> 172.21.15.99:0/872132478 conn(0x7fa3741034e0 msgr2=0x7fa3741040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 shutdown_connections 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:23.193+0000 7fa37b321700 1 -- 172.21.15.99:0/872132478 wait complete. 2024-05-14T16:52:23.800 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (3/15)... 2024-05-14T16:52:24.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: Activating manager daemon x 2024-05-14T16:52:24.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: mgrmap e2: x(active, starting, since 0.00333413s) 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: Manager daemon x is now available 2024-05-14T16:52:24.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:52:24.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:52:24.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' 2024-05-14T16:52:24.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' 2024-05-14T16:52:24.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:24 smithi099 ceph-mon[31851]: from='mgr.14100 172.21.15.99:0/1924931353' entity='mgr.x' 2024-05-14T16:52:25.915 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:25 smithi099 ceph-mon[31851]: mgrmap e3: x(active, since 1.00747s) 2024-05-14T16:52:26.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:26 smithi099 ceph-mon[31851]: mgrmap e4: x(active, since 2s) 2024-05-14T16:52:26.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:26 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3239870350' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:52:27.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:27.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:27.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:52:27.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:52:27.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:52:27.068 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:52:27.068 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:52:27.069 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:52:27.070 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:52:09.109836+0000", 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856eee9700 1 Processor -- start 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856eee9700 1 -- start start 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856eee9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f8568107760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856eee9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8568107d30 con 0x7f8568107340 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856cc85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f8568107760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856cc85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f8568107760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44774/0 (socket says 172.21.15.99:44774) 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856cc85700 1 -- 172.21.15.99:0/2326150816 learned_addr learned my addr 172.21.15.99:0/2326150816 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856cc85700 1 -- 172.21.15.99:0/2326150816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856810a2d0 con 0x7f8568107340 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.209+0000 7f856cc85700 1 --2- 172.21.15.99:0/2326150816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f8568107760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f855c0098a0 tx=0x7f855c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb60ae31234621a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:27.071 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f85677fe700 1 -- 172.21.15.99:0/2326150816 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f855c004160 con 0x7f8568107340 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f85677fe700 1 -- 172.21.15.99:0/2326150816 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f855c004300 con 0x7f8568107340 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f85677fe700 1 -- 172.21.15.99:0/2326150816 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f855c011af0 con 0x7f8568107340 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 -- 172.21.15.99:0/2326150816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 msgr2=0x7f8568107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 --2- 172.21.15.99:0/2326150816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f8568107760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f855c0098a0 tx=0x7f855c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 -- 172.21.15.99:0/2326150816 shutdown_connections 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 --2- 172.21.15.99:0/2326150816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f8568107760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 -- 172.21.15.99:0/2326150816 >> 172.21.15.99:0/2326150816 conn(0x7f8568102cb0 msgr2=0x7f85681050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 -- 172.21.15.99:0/2326150816 shutdown_connections 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.210+0000 7f856eee9700 1 -- 172.21.15.99:0/2326150816 wait complete. 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856eee9700 1 Processor -- start 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856eee9700 1 -- start start 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856eee9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f856819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856eee9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f856810d680 con 0x7f8568107340 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856cc85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f856819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:27.072 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856cc85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f856819d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44790/0 (socket says 172.21.15.99:44790) 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.211+0000 7f856cc85700 1 -- 172.21.15.99:0/3239870350 learned_addr learned my addr 172.21.15.99:0/3239870350 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f856cc85700 1 -- 172.21.15.99:0/3239870350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856819d570 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f856cc85700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f856819d030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f855c009fb0 tx=0x7f855c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f855c0036a0 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f855c01a430 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856819d800 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f855c019550 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.212+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a04f0 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.213+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f855c003840 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.213+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8568064ba0 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.213+0000 7f8565ffb700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f855803e3d0 0x7f8558040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.213+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f855c053800 con 0x7f8568107340 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.213+0000 7f8567fff700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f855803e3d0 0x7f8558040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:27.073 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.214+0000 7f8567fff700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f855803e3d0 0x7f8558040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8554006fd0 tx=0x7f8554006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.219+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f855c017e90 con 0x7f8568107340 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.513+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f85681a06c0 con 0x7f8568107340 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.513+0000 7f8565ffb700 1 -- 172.21.15.99:0/3239870350 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f85681a06c0 con 0x7f8568107340 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f855803e3d0 msgr2=0x7f8558040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f855803e3d0 0x7f8558040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8554006fd0 tx=0x7f8554006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 msgr2=0x7f856819d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f856819d030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f855c009fb0 tx=0x7f855c011940 comp rx=0 tx=0).stop 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 shutdown_connections 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f855803e3d0 0x7f8558040890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 --2- 172.21.15.99:0/3239870350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8568107340 0x7f856819d030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.516+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 >> 172.21.15.99:0/3239870350 conn(0x7f8568102cb0 msgr2=0x7f8568103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.517+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 shutdown_connections 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:26.517+0000 7f856eee9700 1 -- 172.21.15.99:0/3239870350 wait complete. 2024-05-14T16:52:27.074 INFO:teuthology.orchestra.run.smithi099.stdout:mgr is available 2024-05-14T16:52:28.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:27 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4092173217' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T16:52:28.236 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa797fb5700 1 Processor -- start 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa797fb5700 1 -- start start 2024-05-14T16:52:28.237 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa797fb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa7901094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa797fb5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa790109a20 con 0x7fa7901090c0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa795d51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa7901094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa795d51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa7901094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44796/0 (socket says 172.21.15.99:44796) 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa795d51700 1 -- 172.21.15.99:0/3134104126 learned_addr learned my addr 172.21.15.99:0/3134104126 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.465+0000 7fa795d51700 1 -- 172.21.15.99:0/3134104126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa790109ba0 con 0x7fa7901090c0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa795d51700 1 --2- 172.21.15.99:0/3134104126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa7901094e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0098a0 tx=0x7fa78c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a87bfdaa26ae32c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa794d4f700 1 -- 172.21.15.99:0/3134104126 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa78c004160 con 0x7fa7901090c0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa794d4f700 1 -- 172.21.15.99:0/3134104126 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa78c004300 con 0x7fa7901090c0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa794d4f700 1 -- 172.21.15.99:0/3134104126 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa78c011af0 con 0x7fa7901090c0 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa797fb5700 1 -- 172.21.15.99:0/3134104126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 msgr2=0x7fa7901094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa797fb5700 1 --2- 172.21.15.99:0/3134104126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa7901094e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0098a0 tx=0x7fa78c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa797fb5700 1 -- 172.21.15.99:0/3134104126 shutdown_connections 2024-05-14T16:52:28.238 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa797fb5700 1 --2- 172.21.15.99:0/3134104126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa7901094e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.466+0000 7fa797fb5700 1 -- 172.21.15.99:0/3134104126 >> 172.21.15.99:0/3134104126 conn(0x7fa790104720 msgr2=0x7fa790106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa797fb5700 1 -- 172.21.15.99:0/3134104126 shutdown_connections 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa797fb5700 1 -- 172.21.15.99:0/3134104126 wait complete. 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa797fb5700 1 Processor -- start 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa797fb5700 1 -- start start 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa797fb5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa79019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa797fb5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa79010b070 con 0x7fa7901090c0 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa795d51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa79019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa795d51700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa79019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44800/0 (socket says 172.21.15.99:44800) 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.467+0000 7fa795d51700 1 -- 172.21.15.99:0/4092173217 learned_addr learned my addr 172.21.15.99:0/4092173217 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa795d51700 1 -- 172.21.15.99:0/4092173217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79019d490 con 0x7fa7901090c0 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa795d51700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa79019cf50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa78c003f40 tx=0x7fa78c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa78c0038b0 con 0x7fa7901090c0 2024-05-14T16:52:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79019d720 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa78c003a50 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa78c01a5c0 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.468+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7901a0410 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.469+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa790064ba0 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.469+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa78c024430 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.469+0000 7fa786ffd700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fa77c03e3d0 0x7fa77c040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.469+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa78c053a70 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.469+0000 7fa795550700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fa77c03e3d0 0x7fa77c040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.470+0000 7fa795550700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fa77c03e3d0 0x7fa77c040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa780006fd0 tx=0x7fa780006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.474+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa78c00b3e0 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.674+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa7901a05e0 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.674+0000 7fa786ffd700 1 -- 172.21.15.99:0/4092173217 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fa7901a05e0 con 0x7fa7901090c0 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fa77c03e3d0 msgr2=0x7fa77c040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:28.240 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fa77c03e3d0 0x7fa77c040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa780006fd0 tx=0x7fa780006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 msgr2=0x7fa79019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa79019cf50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa78c003f40 tx=0x7fa78c011890 comp rx=0 tx=0).stop 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 shutdown_connections 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fa77c03e3d0 0x7fa77c040890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 --2- 172.21.15.99:0/4092173217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7901090c0 0x7fa79019cf50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 >> 172.21.15.99:0/4092173217 conn(0x7fa790104720 msgr2=0x7fa790105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 shutdown_connections 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:27.677+0000 7fa797fb5700 1 -- 172.21.15.99:0/4092173217 wait complete. 2024-05-14T16:52:28.241 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling cephadm module... 2024-05-14T16:52:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:28 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2324735660' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-14T16:52:29.265 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: ignoring --setuser ceph since I am not root 2024-05-14T16:52:29.265 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: ignoring --setgroup ceph since I am not root 2024-05-14T16:52:29.265 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:29.157+0000 7f0276b70200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:52:29.645 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:29.344+0000 7f0276b70200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:52:29.645 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:29.448+0000 7f0276b70200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9f545700 1 Processor -- start 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9f545700 1 -- start start 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9f545700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9810bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9f545700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9807ce60 con 0x7f6b98109af0 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9d2e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9810bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9d2e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9810bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44810/0 (socket says 172.21.15.99:44810) 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.673+0000 7f6b9d2e1700 1 -- 172.21.15.99:0/1296306906 learned_addr learned my addr 172.21.15.99:0/1296306906 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b9d2e1700 1 -- 172.21.15.99:0/1296306906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9810c420 con 0x7f6b98109af0 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b9d2e1700 1 --2- 172.21.15.99:0/1296306906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9810bee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b940098a0 tx=0x7f6b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d233a74ab18ce323 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b8ffff700 1 -- 172.21.15.99:0/1296306906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b94004160 con 0x7f6b98109af0 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b8ffff700 1 -- 172.21.15.99:0/1296306906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b94004300 con 0x7f6b98109af0 2024-05-14T16:52:29.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b8ffff700 1 -- 172.21.15.99:0/1296306906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b94011af0 con 0x7f6b98109af0 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b9f545700 1 -- 172.21.15.99:0/1296306906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 msgr2=0x7f6b9810bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.674+0000 7f6b9f545700 1 --2- 172.21.15.99:0/1296306906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9810bee0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b940098a0 tx=0x7f6b94009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 -- 172.21.15.99:0/1296306906 shutdown_connections 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 --2- 172.21.15.99:0/1296306906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9810bee0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 -- 172.21.15.99:0/1296306906 >> 172.21.15.99:0/1296306906 conn(0x7f6b981034c0 msgr2=0x7f6b98105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 -- 172.21.15.99:0/1296306906 shutdown_connections 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 -- 172.21.15.99:0/1296306906 wait complete. 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 Processor -- start 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 -- start start 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.675+0000 7f6b9f545700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9810d3e0 con 0x7f6b98109af0 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9d2e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9d2e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44824/0 (socket says 172.21.15.99:44824) 2024-05-14T16:52:29.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9d2e1700 1 -- 172.21.15.99:0/2324735660 learned_addr learned my addr 172.21.15.99:0/2324735660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9d2e1700 1 -- 172.21.15.99:0/2324735660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9819d520 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9d2e1700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9819cfe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009fb0 tx=0x7f6b94011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b940036a0 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9819d7b0 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b9401a430 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b94019550 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.676+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b981a04a0 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.677+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f6b9401a8e0 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.677+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b98064ba0 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.677+0000 7f6b8e7fc700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6b8403e3d0 0x7f6b84040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.678+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6b940537e0 con 0x7f6b98109af0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.678+0000 7f6b9cae0700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6b8403e3d0 0x7f6b84040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.678+0000 7f6b9cae0700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6b8403e3d0 0x7f6b84040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b88006fd0 tx=0x7f6b88006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:29.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.682+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b9400b3e0 con 0x7f6b98109af0 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.939+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6b981a06c0 con 0x7f6b98109af0 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.990+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6b981a06c0 con 0x7f6b98109af0 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.991+0000 7f6b8e7fc700 1 -- 172.21.15.99:0/2324735660 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f6b94003840 con 0x7f6b98109af0 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6b8403e3d0 msgr2=0x7f6b84040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6b8403e3d0 0x7f6b84040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b88006fd0 tx=0x7f6b88006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 msgr2=0x7f6b9819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9819cfe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009fb0 tx=0x7f6b94011940 comp rx=0 tx=0).stop 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 shutdown_connections 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6b8403e3d0 0x7f6b84040890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 --2- 172.21.15.99:0/2324735660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b98109af0 0x7f6b9819cfe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.993+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 >> 172.21.15.99:0/2324735660 conn(0x7f6b981034c0 msgr2=0x7f6b98104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.994+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 shutdown_connections 2024-05-14T16:52:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:28.994+0000 7f6b9f545700 1 -- 172.21.15.99:0/2324735660 wait complete. 2024-05-14T16:52:29.895 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:29.762+0000 7f0276b70200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:52:29.895 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:29.850+0000 7f0276b70200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:52:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:29 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2324735660' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-14T16:52:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:29 smithi099 ceph-mon[31851]: mgrmap e5: x(active, since 5s) 2024-05-14T16:52:30.935 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:30.935 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-14T16:52:30.935 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:52:30.935 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc39e3a700 1 Processor -- start 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc39e3a700 1 -- start start 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc39e3a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3410bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc39e3a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3407ce60 con 0x7fcc34109af0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3410bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3410bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44844/0 (socket says 172.21.15.99:44844) 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc337fe700 1 -- 172.21.15.99:0/2019118432 learned_addr learned my addr 172.21.15.99:0/2019118432 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.082+0000 7fcc337fe700 1 -- 172.21.15.99:0/2019118432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3410c420 con 0x7fcc34109af0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc337fe700 1 --2- 172.21.15.99:0/2019118432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3410bee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcc240098a0 tx=0x7fcc24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15805352e1bbe6b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc327fc700 1 -- 172.21.15.99:0/2019118432 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc24004160 con 0x7fcc34109af0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc327fc700 1 -- 172.21.15.99:0/2019118432 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc24004300 con 0x7fcc34109af0 2024-05-14T16:52:30.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc327fc700 1 -- 172.21.15.99:0/2019118432 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc24011af0 con 0x7fcc34109af0 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 -- 172.21.15.99:0/2019118432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 msgr2=0x7fcc3410bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 --2- 172.21.15.99:0/2019118432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3410bee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcc240098a0 tx=0x7fcc24009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 -- 172.21.15.99:0/2019118432 shutdown_connections 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 --2- 172.21.15.99:0/2019118432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3410bee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 -- 172.21.15.99:0/2019118432 >> 172.21.15.99:0/2019118432 conn(0x7fcc34103500 msgr2=0x7fcc34105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 -- 172.21.15.99:0/2019118432 shutdown_connections 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.083+0000 7fcc39e3a700 1 -- 172.21.15.99:0/2019118432 wait complete. 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc39e3a700 1 Processor -- start 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc39e3a700 1 -- start start 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc39e3a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc39e3a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3410d3e0 con 0x7fcc34109af0 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc337fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44854/0 (socket says 172.21.15.99:44854) 2024-05-14T16:52:30.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.084+0000 7fcc337fe700 1 -- 172.21.15.99:0/1729233441 learned_addr learned my addr 172.21.15.99:0/1729233441 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc337fe700 1 -- 172.21.15.99:0/1729233441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3419d490 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc337fe700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3419cf50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc24003f40 tx=0x7fcc24011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc240038b0 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc24003a50 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3419d720 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc2401a5c0 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.085+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341a0410 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.086+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fcc2401a8c0 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.086+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc34064ba0 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.086+0000 7fcc30ff9700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 0x7fcc1c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.086+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc24053a40 con 0x7fcc34109af0 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.086+0000 7fcc32ffd700 1 -- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 msgr2=0x7fcc1c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.086+0000 7fcc32ffd700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 0x7fcc1c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:52:30.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.091+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc2400b3e0 con 0x7fcc34109af0 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.287+0000 7fcc32ffd700 1 -- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 msgr2=0x7fcc1c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.287+0000 7fcc32ffd700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 0x7fcc1c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.394+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcc341a05e0 con 0x7fcc34109af0 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.394+0000 7fcc30ff9700 1 -- 172.21.15.99:0/1729233441 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcc341a05e0 con 0x7fcc34109af0 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 msgr2=0x7fcc1c0408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 0x7fcc1c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 msgr2=0x7fcc3419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:30.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3419cf50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc24003f40 tx=0x7fcc24011890 comp rx=0 tx=0).stop 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 shutdown_connections 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7fcc1c03e420 0x7fcc1c0408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 --2- 172.21.15.99:0/1729233441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc34109af0 0x7fcc3419cf50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.396+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 >> 172.21.15.99:0/1729233441 conn(0x7fcc34103500 msgr2=0x7fcc34104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.397+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 shutdown_connections 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:30.397+0000 7fcc39e3a700 1 -- 172.21.15.99:0/1729233441 wait complete. 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-05-14T16:52:30.940 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 5... 2024-05-14T16:52:31.262 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1729233441' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T16:52:31.262 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:31.088+0000 7f0276b70200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:52:31.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:31.262+0000 7f0276b70200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:52:31.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:31.440+0000 7f0276b70200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:52:31.973 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:31.729+0000 7f0276b70200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:52:31.974 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:31.822+0000 7f0276b70200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:52:32.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:32.023+0000 7f0276b70200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:52:32.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:32.216+0000 7f0276b70200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:52:33.035 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:32.812+0000 7f0276b70200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:52:33.035 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:33.034+0000 7f0276b70200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:52:33.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:33.458+0000 7f0276b70200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:52:33.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:33.555+0000 7f0276b70200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:52:34.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:33.924+0000 7f0276b70200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:52:34.617 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:34.280+0000 7f0276b70200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:52:34.617 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:34.372+0000 7f0276b70200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:52:35.014 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:34.616+0000 7f0276b70200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:52:35.014 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:34.749+0000 7f0276b70200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:52:35.014 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:34.844+0000 7f0276b70200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:52:35.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:35.170+0000 7f0276b70200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:52:35.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:35.262+0000 7f0276b70200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:52:36.139 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:35.866+0000 7f0276b70200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:52:36.139 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:36.086+0000 7f0276b70200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:52:36.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: Active manager daemon x restarted 2024-05-14T16:52:36.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: Activating manager daemon x 2024-05-14T16:52:36.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: osdmap e2: 0 total, 0 up, 0 in 2024-05-14T16:52:36.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: mgrmap e6: x(active, starting, since 0.00326616s) 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: Manager daemon x is now available 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:36.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:36 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:52:37.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:37 smithi099 ceph-mon[31851]: Found migration_current of "None". Setting to last migration. 2024-05-14T16:52:37.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:37 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:52:37.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:37 smithi099 ceph-mon[31851]: mgrmap e7: x(active, since 1.00571s) 2024-05-14T16:52:37.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:37 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:37.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:37 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.366+0000 7f6efbb8f700 1 Processor -- start 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.366+0000 7f6efbb8f700 1 -- start start 2024-05-14T16:52:37.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.366+0000 7f6efbb8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef407d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.366+0000 7f6efbb8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef407d8e0 con 0x7f6ef407ee60 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef992b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef407d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef992b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef407d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44870/0 (socket says 172.21.15.99:44870) 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef992b700 1 -- 172.21.15.99:0/1594402674 learned_addr learned my addr 172.21.15.99:0/1594402674 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef992b700 1 -- 172.21.15.99:0/1594402674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef407da60 con 0x7f6ef407ee60 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef992b700 1 --2- 172.21.15.99:0/1594402674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef407d310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40098a0 tx=0x7f6ee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95399e439ece2c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef8929700 1 -- 172.21.15.99:0/1594402674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee4004160 con 0x7f6ef407ee60 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef8929700 1 -- 172.21.15.99:0/1594402674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ee4004300 con 0x7f6ef407ee60 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.367+0000 7f6ef8929700 1 -- 172.21.15.99:0/1594402674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee4011af0 con 0x7f6ef407ee60 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 -- 172.21.15.99:0/1594402674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 msgr2=0x7f6ef407d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/1594402674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef407d310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40098a0 tx=0x7f6ee4009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 -- 172.21.15.99:0/1594402674 shutdown_connections 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/1594402674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef407d310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 -- 172.21.15.99:0/1594402674 >> 172.21.15.99:0/1594402674 conn(0x7f6ef4104620 msgr2=0x7f6ef4106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 -- 172.21.15.99:0/1594402674 shutdown_connections 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.368+0000 7f6efbb8f700 1 -- 172.21.15.99:0/1594402674 wait complete. 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6efbb8f700 1 Processor -- start 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6efbb8f700 1 -- start start 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6efbb8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6efbb8f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef410f590 con 0x7f6ef407ee60 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6ef992b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:37.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6ef992b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44872/0 (socket says 172.21.15.99:44872) 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6ef992b700 1 -- 172.21.15.99:0/3618900542 learned_addr learned my addr 172.21.15.99:0/3618900542 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6ef992b700 1 -- 172.21.15.99:0/3618900542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef419f6b0 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.369+0000 7f6ef992b700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef419f170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011ac0 tx=0x7f6ee4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.370+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee4003b20 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.370+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ee401a8c0 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.370+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef419f940 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.370+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee40248d0 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.370+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef41a2630 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.371+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f6ee4022020 con 0x7f6ef407ee60 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.371+0000 7f6eeaffd700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.371+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 --> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ee0041040 con 0x7f6ee003e470 2024-05-14T16:52:37.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.371+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ee4053520 con 0x7f6ef407ee60 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.371+0000 7f6ef912a700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 msgr2=0x7f6ee0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.371+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.572+0000 7f6ef912a700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 msgr2=0x7f6ee0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.572+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.973+0000 7f6ef912a700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 msgr2=0x7f6ee0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:31.973+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:32.774+0000 7f6ef912a700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 msgr2=0x7f6ee0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:32.774+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:34.375+0000 7f6ef912a700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 msgr2=0x7f6ee0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1504956360 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:34.375+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T16:52:37.697 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:36.089+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f6ee401a430 con 0x7f6ef407ee60 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:36.089+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 msgr2=0x7f6ee0040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:36.089+0000 7f6eeaffd700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.090+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f6ee4018a40 con 0x7f6ef407ee60 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.091+0000 7f6eeaffd700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f6ee0041d60 0x7f6ee0044150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.091+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ee0041040 con 0x7f6ee0041d60 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.091+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f6ee0041d60 0x7f6ee0044150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.092+0000 7f6ef912a700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f6ee0041d60 0x7f6ee0044150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0003a10 tx=0x7f6ef0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.092+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6ee0041040 con 0x7f6ee0041d60 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.100+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6ef406a6a0 con 0x7f6ee0041d60 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6eeaffd700 1 -- 172.21.15.99:0/3618900542 <== mgr.14120 v2:172.21.15.99:6800/1816772983 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6ef406a6a0 con 0x7f6ee0041d60 2024-05-14T16:52:37.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f6ee0041d60 msgr2=0x7f6ee0044150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f6ee0041d60 0x7f6ee0044150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0003a10 tx=0x7f6ef0009290 comp rx=0 tx=0).stop 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 msgr2=0x7f6ef419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef419f170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011ac0 tx=0x7f6ee4011ba0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 shutdown_connections 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f6ee0041d60 0x7f6ee0044150 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ef407ee60 0x7f6ef419f170 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 --2- 172.21.15.99:0/3618900542 >> [v2:172.21.15.99:6800/1504956360,v1:172.21.15.99:6801/1504956360] conn(0x7f6ee003e470 0x7f6ee0040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 >> 172.21.15.99:0/3618900542 conn(0x7f6ef4104620 msgr2=0x7f6ef4105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 shutdown_connections 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:37.101+0000 7f6efbb8f700 1 -- 172.21.15.99:0/3618900542 wait complete. 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 5 is available 2024-05-14T16:52:37.699 INFO:teuthology.orchestra.run.smithi099.stdout:Setting orchestrator backend to cephadm... 2024-05-14T16:52:38.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T16:52:38.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T16:52:38.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: [14/May/2024:16:52:37] ENGINE Bus STARTING 2024-05-14T16:52:38.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: [14/May/2024:16:52:37] ENGINE Serving on http://172.21.15.99:8765 2024-05-14T16:52:38.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: [14/May/2024:16:52:37] ENGINE Serving on https://172.21.15.99:7150 2024-05-14T16:52:38.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: [14/May/2024:16:52:37] ENGINE Bus STARTED 2024-05-14T16:52:38.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:38 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f0562397700 1 Processor -- start 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f0562397700 1 -- start start 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f0562397700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f0562397700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1099b0 con 0x7f055c109050 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f055bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f055bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44496/0 (socket says 172.21.15.99:44496) 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f055bfff700 1 -- 172.21.15.99:0/2944026742 learned_addr learned my addr 172.21.15.99:0/2944026742 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:38.897 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.086+0000 7f055bfff700 1 -- 172.21.15.99:0/2944026742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c109b30 con 0x7f055c109050 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f055bfff700 1 --2- 172.21.15.99:0/2944026742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c109470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f054c0098a0 tx=0x7f054c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de52ae479318ad5a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f055affd700 1 -- 172.21.15.99:0/2944026742 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f054c004160 con 0x7f055c109050 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f055affd700 1 -- 172.21.15.99:0/2944026742 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f054c004300 con 0x7f055c109050 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f055affd700 1 -- 172.21.15.99:0/2944026742 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f054c011af0 con 0x7f055c109050 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 -- 172.21.15.99:0/2944026742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 msgr2=0x7f055c109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 --2- 172.21.15.99:0/2944026742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c109470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f054c0098a0 tx=0x7f054c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 -- 172.21.15.99:0/2944026742 shutdown_connections 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 --2- 172.21.15.99:0/2944026742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c109470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 -- 172.21.15.99:0/2944026742 >> 172.21.15.99:0/2944026742 conn(0x7f055c104670 msgr2=0x7f055c106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 -- 172.21.15.99:0/2944026742 shutdown_connections 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.087+0000 7f0562397700 1 -- 172.21.15.99:0/2944026742 wait complete. 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.088+0000 7f0562397700 1 Processor -- start 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.088+0000 7f0562397700 1 -- start start 2024-05-14T16:52:38.898 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.088+0000 7f0562397700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.088+0000 7f0562397700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10b050 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f055bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f055bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44510/0 (socket says 172.21.15.99:44510) 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f055bfff700 1 -- 172.21.15.99:0/2971367345 learned_addr learned my addr 172.21.15.99:0/2971367345 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f055bfff700 1 -- 172.21.15.99:0/2971367345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c19d4f0 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f055bfff700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c19cfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f054c011ac0 tx=0x7f054c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f054c003970 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f054c003b10 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c19d780 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f054c024610 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.089+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a0470 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.090+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f054c01a9e0 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.090+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c064ba0 con 0x7f055c109050 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.090+0000 7f05597fa700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f054403e3a0 0x7f0544040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:38.899 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.090+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f054c0536e0 con 0x7f055c109050 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.091+0000 7f055b7fe700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f054403e3a0 0x7f0544040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.091+0000 7f055b7fe700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f054403e3a0 0x7f0544040860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0550006fd0 tx=0x7f0550006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.096+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f054c0247f0 con 0x7f055c109050 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.142+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f054c055bf0 con 0x7f055c109050 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.320+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f055c1058e0 con 0x7f054403e3a0 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.325+0000 7f05597fa700 1 -- 172.21.15.99:0/2971367345 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f055c1058e0 con 0x7f054403e3a0 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f054403e3a0 msgr2=0x7f0544040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f054403e3a0 0x7f0544040860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0550006fd0 tx=0x7f0550006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 msgr2=0x7f055c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c19cfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f054c011ac0 tx=0x7f054c011ba0 comp rx=0 tx=0).stop 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 shutdown_connections 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f054403e3a0 0x7f0544040860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 --2- 172.21.15.99:0/2971367345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f055c109050 0x7f055c19cfb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 >> 172.21.15.99:0/2971367345 conn(0x7f055c104670 msgr2=0x7f055c1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:38.900 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 shutdown_connections 2024-05-14T16:52:38.901 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:38.328+0000 7f0562397700 1 -- 172.21.15.99:0/2971367345 wait complete. 2024-05-14T16:52:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:39 smithi099 ceph-mon[31851]: mgrmap e8: x(active, since 2s) 2024-05-14T16:52:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:39 smithi099 ceph-mon[31851]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:39 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:39 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca39933700 1 Processor -- start 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca39933700 1 -- start start 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca39933700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca3410bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca39933700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca3407ce00 con 0x7fca34109b40 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca32ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca3410bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca32ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca3410bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44518/0 (socket says 172.21.15.99:44518) 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca32ffd700 1 -- 172.21.15.99:0/4177664346 learned_addr learned my addr 172.21.15.99:0/4177664346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:40.121 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca32ffd700 1 -- 172.21.15.99:0/4177664346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3410c470 con 0x7fca34109b40 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca32ffd700 1 --2- 172.21.15.99:0/4177664346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca3410bf30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca240098a0 tx=0x7fca24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6194cd0759ef855c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.340+0000 7fca31ffb700 1 -- 172.21.15.99:0/4177664346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca24004160 con 0x7fca34109b40 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca31ffb700 1 -- 172.21.15.99:0/4177664346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fca24004300 con 0x7fca34109b40 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca31ffb700 1 -- 172.21.15.99:0/4177664346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca24011af0 con 0x7fca34109b40 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 -- 172.21.15.99:0/4177664346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 msgr2=0x7fca3410bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 --2- 172.21.15.99:0/4177664346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca3410bf30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca240098a0 tx=0x7fca24009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 -- 172.21.15.99:0/4177664346 shutdown_connections 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 --2- 172.21.15.99:0/4177664346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca3410bf30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 -- 172.21.15.99:0/4177664346 >> 172.21.15.99:0/4177664346 conn(0x7fca341034f0 msgr2=0x7fca34105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 -- 172.21.15.99:0/4177664346 shutdown_connections 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.341+0000 7fca39933700 1 -- 172.21.15.99:0/4177664346 wait complete. 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca39933700 1 Processor -- start 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca39933700 1 -- start start 2024-05-14T16:52:40.122 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca39933700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca341093c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca39933700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca3410d3d0 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca32ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca341093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca32ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca341093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44520/0 (socket says 172.21.15.99:44520) 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca32ffd700 1 -- 172.21.15.99:0/4016598070 learned_addr learned my addr 172.21.15.99:0/4016598070 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.342+0000 7fca32ffd700 1 -- 172.21.15.99:0/4016598070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca34109900 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.343+0000 7fca32ffd700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca341093c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca24009fb0 tx=0x7fca24011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.343+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca240036a0 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.343+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fca2401a430 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.343+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca24019550 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.343+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca34107af0 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.343+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca34108030 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.344+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fca24019800 con 0x7fca34109b40 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.344+0000 7fca38931700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fca1c03e470 0x7fca1c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:40.123 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.344+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3406a6a0 con 0x7fca34109b40 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.344+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fca24053b80 con 0x7fca34109b40 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.344+0000 7fca327fc700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fca1c03e470 0x7fca1c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.344+0000 7fca327fc700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fca1c03e470 0x7fca1c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca28006fd0 tx=0x7fca28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.349+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca2401a5e0 con 0x7fca34109b40 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.555+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fca341089b0 con 0x7fca1c03e470 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.556+0000 7fca38931700 1 -- 172.21.15.99:0/4016598070 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fca341089b0 con 0x7fca1c03e470 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.558+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fca1c03e470 msgr2=0x7fca1c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.558+0000 7fca39933700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fca1c03e470 0x7fca1c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca28006fd0 tx=0x7fca28006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.558+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 msgr2=0x7fca341093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.558+0000 7fca39933700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca341093c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca24009fb0 tx=0x7fca24011940 comp rx=0 tx=0).stop 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.558+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 shutdown_connections 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.558+0000 7fca39933700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fca1c03e470 0x7fca1c040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.559+0000 7fca39933700 1 --2- 172.21.15.99:0/4016598070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34109b40 0x7fca341093c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:40.124 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.559+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 >> 172.21.15.99:0/4016598070 conn(0x7fca341034f0 msgr2=0x7fca34104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:40.125 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.559+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 shutdown_connections 2024-05-14T16:52:40.125 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:39.559+0000 7fca39933700 1 -- 172.21.15.99:0/4016598070 wait complete. 2024-05-14T16:52:40.125 INFO:teuthology.orchestra.run.smithi099.stdout:Generating ssh key... 2024-05-14T16:52:40.415 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:40 smithi099 ceph-mon[31851]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:41 smithi099 ceph-mon[31851]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:41 smithi099 ceph-mon[31851]: Generating ssh key... 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: Generating public/private rsa key pair. 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: Your identification has been saved in /tmp/tmp71jualil/key. 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: Your public key has been saved in /tmp/tmp71jualil/key.pub. 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: The key fingerprint is: 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: SHA256:4GxK4xnj2b2/azC6MDDqQyJ2gDJKfB3pd4JtTsurPbs ceph-2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: The key's randomart image is: 2024-05-14T16:52:41.461 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: +---[RSA 3072]----+ 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: | . | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: | o | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: |o o = | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: |++ . * B . | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: |+.= = X S | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: |++ B X =o | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: |* . O ..oo | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: |.. ooo .. | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: | .. .oE=o+o | 2024-05-14T16:52:41.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: +----[SHA256]-----+ 2024-05-14T16:52:41.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.518+0000 7f8aac781700 1 Processor -- start 2024-05-14T16:52:41.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.518+0000 7f8aac781700 1 -- start start 2024-05-14T16:52:41.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aac781700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa4108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aac781700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4108890 con 0x7f8aa4107f10 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aaa51d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa4108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aaa51d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa4108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44532/0 (socket says 172.21.15.99:44532) 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aaa51d700 1 -- 172.21.15.99:0/2365179938 learned_addr learned my addr 172.21.15.99:0/2365179938 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aaa51d700 1 -- 172.21.15.99:0/2365179938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa41090f0 con 0x7f8aa4107f10 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aaa51d700 1 --2- 172.21.15.99:0/2365179938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa4108350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a940098a0 tx=0x7f8a94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58a7532d402c7be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aa951b700 1 -- 172.21.15.99:0/2365179938 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a94004160 con 0x7f8aa4107f10 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aa951b700 1 -- 172.21.15.99:0/2365179938 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a94004300 con 0x7f8aa4107f10 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.519+0000 7f8aa951b700 1 -- 172.21.15.99:0/2365179938 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a94011af0 con 0x7f8aa4107f10 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 -- 172.21.15.99:0/2365179938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 msgr2=0x7f8aa4108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 --2- 172.21.15.99:0/2365179938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa4108350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a940098a0 tx=0x7f8a94009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 -- 172.21.15.99:0/2365179938 shutdown_connections 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 --2- 172.21.15.99:0/2365179938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa4108350 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 -- 172.21.15.99:0/2365179938 >> 172.21.15.99:0/2365179938 conn(0x7f8aa41034e0 msgr2=0x7f8aa4105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 -- 172.21.15.99:0/2365179938 shutdown_connections 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.520+0000 7f8aac781700 1 -- 172.21.15.99:0/2365179938 wait complete. 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aac781700 1 Processor -- start 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aac781700 1 -- start start 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aac781700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aac781700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4109f80 con 0x7f8aa4107f10 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aaa51d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aaa51d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa419d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44546/0 (socket says 172.21.15.99:44546) 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aaa51d700 1 -- 172.21.15.99:0/3319291372 learned_addr learned my addr 172.21.15.99:0/3319291372 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aaa51d700 1 -- 172.21.15.99:0/3319291372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa419d5a0 con 0x7f8aa4107f10 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.521+0000 7f8aaa51d700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa419d060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a940117b0 tx=0x7f8a94011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a94003af0 con 0x7f8aa4107f10 2024-05-14T16:52:41.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a94020070 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa419d830 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a94019630 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41a0520 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8a940198a0 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.522+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa4064ba0 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.523+0000 7f8a9b7fe700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f8a9003e4c0 0x7f8a90040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.523+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a94053c60 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.523+0000 7f8aa9d1c700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f8a9003e4c0 0x7f8a90040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.523+0000 7f8aa9d1c700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f8a9003e4c0 0x7f8a90040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0006fd0 tx=0x7f8aa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.528+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a940535f0 con 0x7f8aa4107f10 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:40.735+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8aa40008d0 con 0x7f8a9003e4c0 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.169+0000 7f8a9b7fe700 1 -- 172.21.15.99:0/3319291372 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8aa40008d0 con 0x7f8a9003e4c0 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f8a9003e4c0 msgr2=0x7f8a90040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f8a9003e4c0 0x7f8a90040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0006fd0 tx=0x7f8aa0006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:41.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 msgr2=0x7f8aa419d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa419d060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a940117b0 tx=0x7f8a94011b70 comp rx=0 tx=0).stop 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 shutdown_connections 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f8a9003e4c0 0x7f8a90040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 --2- 172.21.15.99:0/3319291372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa4107f10 0x7f8aa419d060 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 >> 172.21.15.99:0/3319291372 conn(0x7f8aa41034e0 msgr2=0x7f8aa4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 shutdown_connections 2024-05-14T16:52:41.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:41.171+0000 7f8aac781700 1 -- 172.21.15.99:0/3319291372 wait complete. 2024-05-14T16:52:42.373 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:42 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:42.373 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:42 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:42.935 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:52:42.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.157+0000 7f23c42b7700 1 Processor -- start 2024-05-14T16:52:42.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.157+0000 7f23c42b7700 1 -- start start 2024-05-14T16:52:42.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c42b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc107720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:42.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c42b7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc107cf0 con 0x7f23bc107300 2024-05-14T16:52:42.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c2053700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:42.936 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c2053700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44554/0 (socket says 172.21.15.99:44554) 2024-05-14T16:52:42.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c2053700 1 -- 172.21.15.99:0/2643382882 learned_addr learned my addr 172.21.15.99:0/2643382882 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:42.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c2053700 1 -- 172.21.15.99:0/2643382882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc107e70 con 0x7f23bc107300 2024-05-14T16:52:42.937 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c2053700 1 --2- 172.21.15.99:0/2643382882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc107720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0098a0 tx=0x7f23ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b88ba2e0d6cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:42.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c1051700 1 -- 172.21.15.99:0/2643382882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ac004160 con 0x7f23bc107300 2024-05-14T16:52:42.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.158+0000 7f23c1051700 1 -- 172.21.15.99:0/2643382882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f23ac004300 con 0x7f23bc107300 2024-05-14T16:52:42.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c1051700 1 -- 172.21.15.99:0/2643382882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ac011af0 con 0x7f23bc107300 2024-05-14T16:52:42.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 -- 172.21.15.99:0/2643382882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 msgr2=0x7f23bc107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:42.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 --2- 172.21.15.99:0/2643382882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc107720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0098a0 tx=0x7f23ac009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:42.938 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 -- 172.21.15.99:0/2643382882 shutdown_connections 2024-05-14T16:52:42.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 --2- 172.21.15.99:0/2643382882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc107720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:42.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 -- 172.21.15.99:0/2643382882 >> 172.21.15.99:0/2643382882 conn(0x7f23bc102c80 msgr2=0x7f23bc1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:42.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 -- 172.21.15.99:0/2643382882 shutdown_connections 2024-05-14T16:52:42.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.159+0000 7f23c42b7700 1 -- 172.21.15.99:0/2643382882 wait complete. 2024-05-14T16:52:42.939 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c42b7700 1 Processor -- start 2024-05-14T16:52:42.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c42b7700 1 -- start start 2024-05-14T16:52:42.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c42b7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:42.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c42b7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc10b1e0 con 0x7f23bc107300 2024-05-14T16:52:42.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c2053700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:42.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c2053700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44556/0 (socket says 172.21.15.99:44556) 2024-05-14T16:52:42.940 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c2053700 1 -- 172.21.15.99:0/1990259835 learned_addr learned my addr 172.21.15.99:0/1990259835 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c2053700 1 -- 172.21.15.99:0/1990259835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc10a140 con 0x7f23bc107300 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.160+0000 7f23c2053700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc109c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0117b0 tx=0x7f23ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.161+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ac003af0 con 0x7f23bc107300 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.161+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc108330 con 0x7f23bc107300 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.161+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f23ac020070 con 0x7f23bc107300 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.161+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23ac019550 con 0x7f23bc107300 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.161+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc108870 con 0x7f23bc107300 2024-05-14T16:52:42.941 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.161+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f23ac01a430 con 0x7f23bc107300 2024-05-14T16:52:42.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.162+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc064ba0 con 0x7f23bc107300 2024-05-14T16:52:42.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.162+0000 7f23b37fe700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f23a803e470 0x7f23a8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:42.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.162+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f23ac053a40 con 0x7f23bc107300 2024-05-14T16:52:42.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.162+0000 7f23c1852700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f23a803e470 0x7f23a8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:42.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.162+0000 7f23c1852700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f23a803e470 0x7f23a8040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006fd0 tx=0x7f23b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:42.942 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.167+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23ac0042a0 con 0x7f23bc107300 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.373+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f23bc1090f0 con 0x7f23a803e470 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.374+0000 7f23b37fe700 1 -- 172.21.15.99:0/1990259835 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f23bc1090f0 con 0x7f23a803e470 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.376+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f23a803e470 msgr2=0x7f23a8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.376+0000 7f23c42b7700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f23a803e470 0x7f23a8040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006fd0 tx=0x7f23b8006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.376+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 msgr2=0x7f23bc109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.376+0000 7f23c42b7700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc109c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0117b0 tx=0x7f23ac011b70 comp rx=0 tx=0).stop 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.376+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 shutdown_connections 2024-05-14T16:52:42.943 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.377+0000 7f23c42b7700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f23a803e470 0x7f23a8040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.377+0000 7f23c42b7700 1 --2- 172.21.15.99:0/1990259835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc107300 0x7f23bc109c00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.377+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 >> 172.21.15.99:0/1990259835 conn(0x7f23bc102c80 msgr2=0x7f23bc103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.377+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 shutdown_connections 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:42.377+0000 7f23c42b7700 1 -- 172.21.15.99:0/1990259835 wait complete. 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:Adding key to root@localhost authorized_keys... 2024-05-14T16:52:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:Adding host smithi099... 2024-05-14T16:52:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:43 smithi099 ceph-mon[31851]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:44.454 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:44 smithi099 ceph-mon[31851]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:45 smithi099 ceph-mon[31851]: Deploying cephadm binary to smithi099 2024-05-14T16:52:48.254 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Added host 'smithi099' with addr '172.21.15.99' 2024-05-14T16:52:48.254 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59b6117700 1 Processor -- start 2024-05-14T16:52:48.254 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59b6117700 1 -- start start 2024-05-14T16:52:48.254 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59b6117700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b01094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.254 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59b6117700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01099e0 con 0x7f59b0109080 2024-05-14T16:52:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b01094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b01094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44562/0 (socket says 172.21.15.99:44562) 2024-05-14T16:52:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59af7fe700 1 -- 172.21.15.99:0/1835795441 learned_addr learned my addr 172.21.15.99:0/1835795441 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.385+0000 7f59af7fe700 1 -- 172.21.15.99:0/1835795441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b0109b60 con 0x7f59b0109080 2024-05-14T16:52:48.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59af7fe700 1 --2- 172.21.15.99:0/1835795441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b01094a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59a00098a0 tx=0x7f59a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18197e7206e174 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.260 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59ae7fc700 1 -- 172.21.15.99:0/1835795441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a0004160 con 0x7f59b0109080 2024-05-14T16:52:48.260 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59ae7fc700 1 -- 172.21.15.99:0/1835795441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f59a0004300 con 0x7f59b0109080 2024-05-14T16:52:48.260 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59ae7fc700 1 -- 172.21.15.99:0/1835795441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a0011af0 con 0x7f59b0109080 2024-05-14T16:52:48.260 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 -- 172.21.15.99:0/1835795441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 msgr2=0x7f59b01094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.264 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 --2- 172.21.15.99:0/1835795441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b01094a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59a00098a0 tx=0x7f59a0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.264 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 -- 172.21.15.99:0/1835795441 shutdown_connections 2024-05-14T16:52:48.264 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 --2- 172.21.15.99:0/1835795441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b01094a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.264 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 -- 172.21.15.99:0/1835795441 >> 172.21.15.99:0/1835795441 conn(0x7f59b0104720 msgr2=0x7f59b0106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:48.264 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 -- 172.21.15.99:0/1835795441 shutdown_connections 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.386+0000 7f59b6117700 1 -- 172.21.15.99:0/1835795441 wait complete. 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.387+0000 7f59b6117700 1 Processor -- start 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.387+0000 7f59b6117700 1 -- start start 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.387+0000 7f59b6117700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.387+0000 7f59b6117700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59b010b080 con 0x7f59b0109080 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.387+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44578/0 (socket says 172.21.15.99:44578) 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59af7fe700 1 -- 172.21.15.99:0/3927778645 learned_addr learned my addr 172.21.15.99:0/3927778645 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59af7fe700 1 -- 172.21.15.99:0/3927778645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b019d4a0 con 0x7f59b0109080 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59af7fe700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b019cf60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59a0003f40 tx=0x7f59a0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a00038b0 con 0x7f59b0109080 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f59a0003a50 con 0x7f59b0109080 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b019d730 con 0x7f59b0109080 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59a001a5c0 con 0x7f59b0109080 2024-05-14T16:52:48.265 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.388+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b01a0420 con 0x7f59b0109080 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.389+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f59a001a8c0 con 0x7f59b0109080 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.389+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b0064ba0 con 0x7f59b0109080 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.389+0000 7f59acff9700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f599c03e420 0x7f599c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.389+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59a0053b70 con 0x7f59b0109080 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.389+0000 7f59aeffd700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f599c03e420 0x7f599c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.390+0000 7f59aeffd700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f599c03e420 0x7f599c0408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5998006fd0 tx=0x7f5998006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.394+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59a000e260 con 0x7f59b0109080 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:43.598+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}) -- 0x7f59b0105920 con 0x7f599c03e420 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.713+0000 7f59acff9700 1 -- 172.21.15.99:0/3927778645 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f59b0105920 con 0x7f599c03e420 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f599c03e420 msgr2=0x7f599c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f599c03e420 0x7f599c0408e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5998006fd0 tx=0x7f5998006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 msgr2=0x7f59b019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b019cf60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59a0003f40 tx=0x7f59a0011890 comp rx=0 tx=0).stop 2024-05-14T16:52:48.266 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 shutdown_connections 2024-05-14T16:52:48.267 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f599c03e420 0x7f599c0408e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.267 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 --2- 172.21.15.99:0/3927778645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59b0109080 0x7f59b019cf60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:48.267 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 >> 172.21.15.99:0/3927778645 conn(0x7f59b0104720 msgr2=0x7f59b0105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:48.267 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 shutdown_connections 2024-05-14T16:52:48.267 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:47.716+0000 7f59b6117700 1 -- 172.21.15.99:0/3927778645 wait complete. 2024-05-14T16:52:48.267 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mon service... 2024-05-14T16:52:48.934 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:48 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:48.934 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:48 smithi099 ceph-mon[31851]: Added host smithi099 2024-05-14T16:52:48.934 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:48 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:49.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-14T16:52:49.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.692+0000 7fdb85507700 1 Processor -- start 2024-05-14T16:52:49.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.692+0000 7fdb85507700 1 -- start start 2024-05-14T16:52:49.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb85507700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb801094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:49.593 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb85507700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80109aa0 con 0x7fdb801090b0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb7effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb801094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb7effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb801094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53072/0 (socket says 172.21.15.99:53072) 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb7effd700 1 -- 172.21.15.99:0/3754797371 learned_addr learned my addr 172.21.15.99:0/3754797371 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb7effd700 1 -- 172.21.15.99:0/3754797371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8010a210 con 0x7fdb801090b0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.693+0000 7fdb7effd700 1 --2- 172.21.15.99:0/3754797371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb801094d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098a0 tx=0x7fdb70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d16292cff7aaff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb7e7fc700 1 -- 172.21.15.99:0/3754797371 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb70004160 con 0x7fdb801090b0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb7e7fc700 1 -- 172.21.15.99:0/3754797371 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdb70004300 con 0x7fdb801090b0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb7e7fc700 1 -- 172.21.15.99:0/3754797371 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb70011af0 con 0x7fdb801090b0 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 -- 172.21.15.99:0/3754797371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 msgr2=0x7fdb801094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 --2- 172.21.15.99:0/3754797371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb801094d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098a0 tx=0x7fdb70009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 -- 172.21.15.99:0/3754797371 shutdown_connections 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 --2- 172.21.15.99:0/3754797371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb801094d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 -- 172.21.15.99:0/3754797371 >> 172.21.15.99:0/3754797371 conn(0x7fdb801046d0 msgr2=0x7fdb80106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:49.594 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 -- 172.21.15.99:0/3754797371 shutdown_connections 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.694+0000 7fdb85507700 1 -- 172.21.15.99:0/3754797371 wait complete. 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb85507700 1 Processor -- start 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb85507700 1 -- start start 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb85507700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb8019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb85507700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010d5c0 con 0x7fdb801090b0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb7effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb8019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb7effd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb8019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53082/0 (socket says 172.21.15.99:53082) 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb7effd700 1 -- 172.21.15.99:0/4203567983 learned_addr learned my addr 172.21.15.99:0/4203567983 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.695+0000 7fdb7effd700 1 -- 172.21.15.99:0/4203567983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8019d4e0 con 0x7fdb801090b0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.696+0000 7fdb7effd700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb8019cfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb70003f40 tx=0x7fdb70011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.696+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb700038b0 con 0x7fdb801090b0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.696+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdb70003a50 con 0x7fdb801090b0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.696+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb7001a5c0 con 0x7fdb801090b0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.696+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8019d770 con 0x7fdb801090b0 2024-05-14T16:52:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.696+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a0460 con 0x7fdb801090b0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.697+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fdb70024430 con 0x7fdb801090b0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.697+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb80064ba0 con 0x7fdb801090b0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.697+0000 7fdb7cff9700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fdb6c03e420 0x7fdb6c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.697+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdb70053a60 con 0x7fdb801090b0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.697+0000 7fdb77fff700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fdb6c03e420 0x7fdb6c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.697+0000 7fdb77fff700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fdb6c03e420 0x7fdb6c0408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb68006fd0 tx=0x7fdb68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.702+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb70029df0 con 0x7fdb801090b0 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.934+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdb80105940 con 0x7fdb6c03e420 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.939+0000 7fdb7cff9700 1 -- 172.21.15.99:0/4203567983 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdb80105940 con 0x7fdb6c03e420 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.941+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fdb6c03e420 msgr2=0x7fdb6c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.941+0000 7fdb85507700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fdb6c03e420 0x7fdb6c0408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb68006fd0 tx=0x7fdb68006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.941+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 msgr2=0x7fdb8019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.941+0000 7fdb85507700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb8019cfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb70003f40 tx=0x7fdb70011890 comp rx=0 tx=0).stop 2024-05-14T16:52:49.596 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.942+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 shutdown_connections 2024-05-14T16:52:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.942+0000 7fdb85507700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fdb6c03e420 0x7fdb6c0408e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.942+0000 7fdb85507700 1 --2- 172.21.15.99:0/4203567983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb801090b0 0x7fdb8019cfa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.942+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 >> 172.21.15.99:0/4203567983 conn(0x7fdb801046d0 msgr2=0x7fdb80105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.942+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 shutdown_connections 2024-05-14T16:52:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:48.942+0000 7fdb85507700 1 -- 172.21.15.99:0/4203567983 wait complete. 2024-05-14T16:52:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mgr service... 2024-05-14T16:52:50.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:49 smithi099 ceph-mon[31851]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:50.116 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:49 smithi099 ceph-mon[31851]: Saving service mon spec with placement count:5 2024-05-14T16:52:50.116 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:49 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.023+0000 7f26b386d700 1 Processor -- start 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b386d700 1 -- start start 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b386d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b386d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac108960 con 0x7f26ac107f70 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b1609700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b1609700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53086/0 (socket says 172.21.15.99:53086) 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b1609700 1 -- 172.21.15.99:0/2791506800 learned_addr learned my addr 172.21.15.99:0/2791506800 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b1609700 1 -- 172.21.15.99:0/2791506800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac109170 con 0x7f26ac107f70 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.024+0000 7f26b1609700 1 --2- 172.21.15.99:0/2791506800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac108390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f26a80098a0 tx=0x7f26a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2271c8e6aeb2acc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26a3fff700 1 -- 172.21.15.99:0/2791506800 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8004160 con 0x7f26ac107f70 2024-05-14T16:52:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26a3fff700 1 -- 172.21.15.99:0/2791506800 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f26a8004300 con 0x7f26ac107f70 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26a3fff700 1 -- 172.21.15.99:0/2791506800 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8011af0 con 0x7f26ac107f70 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 -- 172.21.15.99:0/2791506800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 msgr2=0x7f26ac108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 --2- 172.21.15.99:0/2791506800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac108390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f26a80098a0 tx=0x7f26a8009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 -- 172.21.15.99:0/2791506800 shutdown_connections 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 --2- 172.21.15.99:0/2791506800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac108390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 -- 172.21.15.99:0/2791506800 >> 172.21.15.99:0/2791506800 conn(0x7f26ac103510 msgr2=0x7f26ac105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 -- 172.21.15.99:0/2791506800 shutdown_connections 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.025+0000 7f26b386d700 1 -- 172.21.15.99:0/2791506800 wait complete. 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b386d700 1 Processor -- start 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b386d700 1 -- start start 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b386d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b386d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10c520 con 0x7f26ac107f70 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b1609700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:51.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b1609700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53096/0 (socket says 172.21.15.99:53096) 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b1609700 1 -- 172.21.15.99:0/1980443434 learned_addr learned my addr 172.21.15.99:0/1980443434 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.026+0000 7f26b1609700 1 -- 172.21.15.99:0/1980443434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac19c4c0 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.027+0000 7f26b1609700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac19bf80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009fb0 tx=0x7f26a8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.027+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a80036a0 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.027+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f26a801a430 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.027+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac19c750 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.027+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8019550 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.027+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac19cbb0 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.028+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f26a8019800 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.028+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ac064ba0 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.028+0000 7f26a27fc700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f2698040680 0x7f2698042b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.028+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f26a8053970 con 0x7f26ac107f70 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.028+0000 7f26b0e08700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f2698040680 0x7f2698042b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:51.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.028+0000 7f26b0e08700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f2698040680 0x7f2698042b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f269c006fd0 tx=0x7f269c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.033+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26a8023390 con 0x7f26ac107f70 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.258+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f26ac1047d0 con 0x7f2698040680 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.265+0000 7f26a27fc700 1 -- 172.21.15.99:0/1980443434 <== mgr.14120 v2:172.21.15.99:6800/1816772983 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f26ac1047d0 con 0x7f2698040680 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f2698040680 msgr2=0x7f2698042b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f2698040680 0x7f2698042b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f269c006fd0 tx=0x7f269c006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 msgr2=0x7f26ac19bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac19bf80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009fb0 tx=0x7f26a8011940 comp rx=0 tx=0).stop 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 shutdown_connections 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f2698040680 0x7f2698042b40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 --2- 172.21.15.99:0/1980443434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26ac107f70 0x7f26ac19bf80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 >> 172.21.15.99:0/1980443434 conn(0x7f26ac103510 msgr2=0x7f26ac1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 shutdown_connections 2024-05-14T16:52:51.143 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:50.267+0000 7f26b386d700 1 -- 172.21.15.99:0/1980443434 wait complete. 2024-05-14T16:52:51.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:51 smithi099 ceph-mon[31851]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:51.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:51 smithi099 ceph-mon[31851]: Saving service mgr spec with placement count:2 2024-05-14T16:52:51.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:51 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.537+0000 7f54bdba5700 1 Processor -- start 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.537+0000 7f54bdba5700 1 -- start start 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54bdba5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b810bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54bdba5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54b807ce40 con 0x7f54b8109b60 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b810bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b810bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53098/0 (socket says 172.21.15.99:53098) 2024-05-14T16:52:52.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b77fe700 1 -- 172.21.15.99:0/1133070509 learned_addr learned my addr 172.21.15.99:0/1133070509 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b77fe700 1 -- 172.21.15.99:0/1133070509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b810c490 con 0x7f54b8109b60 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b77fe700 1 --2- 172.21.15.99:0/1133070509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b810bf50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009e80 tx=0x7f54a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3326be4b9c945c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b67fc700 1 -- 172.21.15.99:0/1133070509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54a8017070 con 0x7f54b8109b60 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b67fc700 1 -- 172.21.15.99:0/1133070509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54a800b8c0 con 0x7f54b8109b60 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.538+0000 7f54b67fc700 1 -- 172.21.15.99:0/1133070509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54a8004490 con 0x7f54b8109b60 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 -- 172.21.15.99:0/1133070509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 msgr2=0x7f54b810bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 --2- 172.21.15.99:0/1133070509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b810bf50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009e80 tx=0x7f54a800b330 comp rx=0 tx=0).stop 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 -- 172.21.15.99:0/1133070509 shutdown_connections 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 --2- 172.21.15.99:0/1133070509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b810bf50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 -- 172.21.15.99:0/1133070509 >> 172.21.15.99:0/1133070509 conn(0x7f54b8103570 msgr2=0x7f54b8105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 -- 172.21.15.99:0/1133070509 shutdown_connections 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.539+0000 7f54bdba5700 1 -- 172.21.15.99:0/1133070509 wait complete. 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54bdba5700 1 Processor -- start 2024-05-14T16:52:52.532 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54bdba5700 1 -- start start 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54bdba5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54bdba5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54b810d400 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53106/0 (socket says 172.21.15.99:53106) 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54b77fe700 1 -- 172.21.15.99:0/2737673813 learned_addr learned my addr 172.21.15.99:0/2737673813 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54b77fe700 1 -- 172.21.15.99:0/2737673813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b819d4e0 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54b77fe700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b819cfa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b7b0 tx=0x7f54a800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.540+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54a8020020 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.541+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54a8003d10 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.541+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b819d770 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.541+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54a8017040 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.541+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b81a0460 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.541+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f54a8023020 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.542+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b8064ba0 con 0x7f54b8109b60 2024-05-14T16:52:52.533 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.542+0000 7f54b4ff9700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f54a003e4c0 0x7f54a0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.542+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f54a800f050 con 0x7f54b8109b60 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.542+0000 7f54b6ffd700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f54a003e4c0 0x7f54a0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.542+0000 7f54b6ffd700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f54a003e4c0 0x7f54a0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006fd0 tx=0x7f54ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.547+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54a802a020 con 0x7f54b8109b60 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.753+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f54b81a0b50 con 0x7f54b8109b60 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.758+0000 7f54b4ff9700 1 -- 172.21.15.99:0/2737673813 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f54a801a2d0 con 0x7f54b8109b60 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f54a003e4c0 msgr2=0x7f54a0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f54a003e4c0 0x7f54a0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006fd0 tx=0x7f54ac006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 msgr2=0x7f54b819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b819cfa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b7b0 tx=0x7f54a800be90 comp rx=0 tx=0).stop 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 shutdown_connections 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f54a003e4c0 0x7f54a0040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 --2- 172.21.15.99:0/2737673813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54b8109b60 0x7f54b819cfa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 >> 172.21.15.99:0/2737673813 conn(0x7f54b8103570 msgr2=0x7f54b8104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 shutdown_connections 2024-05-14T16:52:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:51.761+0000 7f54bdba5700 1 -- 172.21.15.99:0/2737673813 wait complete. 2024-05-14T16:52:52.775 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:52 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2737673813' entity='client.admin' 2024-05-14T16:52:52.775 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:52 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:52.775 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:52 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:54.316 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7be492a700 1 Processor -- start 2024-05-14T16:52:54.316 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7be492a700 1 -- start start 2024-05-14T16:52:54.316 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7be492a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be0108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:54.316 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7be492a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0108e30 con 0x7f7be0108440 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7bde59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be0108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7bde59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be0108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53112/0 (socket says 172.21.15.99:53112) 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.957+0000 7f7bde59c700 1 -- 172.21.15.99:0/472987156 learned_addr learned my addr 172.21.15.99:0/472987156 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.958+0000 7f7bde59c700 1 -- 172.21.15.99:0/472987156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be0109160 con 0x7f7be0108440 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.958+0000 7f7bde59c700 1 --2- 172.21.15.99:0/472987156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be0108860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00098a0 tx=0x7f7bd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de76a0f102bfde7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.958+0000 7f7bdd59a700 1 -- 172.21.15.99:0/472987156 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0004160 con 0x7f7be0108440 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.958+0000 7f7bdd59a700 1 -- 172.21.15.99:0/472987156 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd0004300 con 0x7f7be0108440 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.958+0000 7f7bdd59a700 1 -- 172.21.15.99:0/472987156 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0011a20 con 0x7f7be0108440 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 -- 172.21.15.99:0/472987156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 msgr2=0x7f7be0108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 --2- 172.21.15.99:0/472987156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be0108860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00098a0 tx=0x7f7bd0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 -- 172.21.15.99:0/472987156 shutdown_connections 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 --2- 172.21.15.99:0/472987156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be0108860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 -- 172.21.15.99:0/472987156 >> 172.21.15.99:0/472987156 conn(0x7f7be0102c80 msgr2=0x7f7be01050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:54.317 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 -- 172.21.15.99:0/472987156 shutdown_connections 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.959+0000 7f7be492a700 1 -- 172.21.15.99:0/472987156 wait complete. 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7be492a700 1 Processor -- start 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7be492a700 1 -- start start 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7be492a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7be492a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7be010c510 con 0x7f7be0108440 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7bde59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7bde59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53124/0 (socket says 172.21.15.99:53124) 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7bde59c700 1 -- 172.21.15.99:0/1819861881 learned_addr learned my addr 172.21.15.99:0/1819861881 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.960+0000 7f7bde59c700 1 -- 172.21.15.99:0/1819861881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be019d500 con 0x7f7be0108440 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.961+0000 7f7bde59c700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be019cfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0004660 tx=0x7f7bd0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.961+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0003aa0 con 0x7f7be0108440 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.961+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be019d790 con 0x7f7be0108440 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.961+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd001b8f0 con 0x7f7be0108440 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.961+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0023900 con 0x7f7be0108440 2024-05-14T16:52:54.318 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.961+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01a0430 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.962+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7bd0023bc0 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.962+0000 7f7bcf7fe700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f7bc803e470 0x7f7bc8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.962+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7bd0053e60 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.962+0000 7f7bddd9b700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f7bc803e470 0x7f7bc8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.962+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be0064ba0 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.962+0000 7f7bddd9b700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f7bc803e470 0x7f7bc8040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4006fd0 tx=0x7f7bd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:52.967+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7bd0083df0 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.194+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f7be002e720 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.198+0000 7f7bcf7fe700 1 -- 172.21.15.99:0/1819861881 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7bd001f030 con 0x7f7be0108440 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f7bc803e470 msgr2=0x7f7bc8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f7bc803e470 0x7f7bc8040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4006fd0 tx=0x7f7bd4006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 msgr2=0x7f7be019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be019cfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0004660 tx=0x7f7bd0011aa0 comp rx=0 tx=0).stop 2024-05-14T16:52:54.319 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 shutdown_connections 2024-05-14T16:52:54.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f7bc803e470 0x7f7bc8040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:54.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 --2- 172.21.15.99:0/1819861881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7be0108440 0x7f7be019cfc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:54.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.200+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 >> 172.21.15.99:0/1819861881 conn(0x7f7be0102c80 msgr2=0x7f7be0103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:54.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.201+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 shutdown_connections 2024-05-14T16:52:54.320 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:53.201+0000 7f7be492a700 1 -- 172.21.15.99:0/1819861881 wait complete. 2024-05-14T16:52:54.320 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling the dashboard module... 2024-05-14T16:52:54.438 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:54 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1819861881' entity='client.admin' 2024-05-14T16:52:54.438 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:54 smithi099 ceph-mon[31851]: from='mgr.14120 172.21.15.99:0/4229705411' entity='mgr.x' 2024-05-14T16:52:55.378 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:55 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3220287307' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-14T16:52:55.379 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:55 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: ignoring --setuser ceph since I am not root 2024-05-14T16:52:55.379 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:55 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: ignoring --setgroup ceph since I am not root 2024-05-14T16:52:55.379 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:55 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:55.378+0000 7f403b7ff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:52:55.669 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:55 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:55.567+0000 7f403b7ff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:52:55.790 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.747+0000 7fd717ddc700 1 Processor -- start 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.747+0000 7fd717ddc700 1 -- start start 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.747+0000 7fd717ddc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd7101094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.747+0000 7fd717ddc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101099e0 con 0x7fd710109080 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.747+0000 7fd715b78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd7101094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd715b78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd7101094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54694/0 (socket says 172.21.15.99:54694) 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd715b78700 1 -- 172.21.15.99:0/3133536646 learned_addr learned my addr 172.21.15.99:0/3133536646 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd715b78700 1 -- 172.21.15.99:0/3133536646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd710109b60 con 0x7fd710109080 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd715b78700 1 --2- 172.21.15.99:0/3133536646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd7101094a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098a0 tx=0x7fd70c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2377a839fbacf672 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd714b76700 1 -- 172.21.15.99:0/3133536646 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd70c004160 con 0x7fd710109080 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd714b76700 1 -- 172.21.15.99:0/3133536646 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd70c004300 con 0x7fd710109080 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.748+0000 7fd714b76700 1 -- 172.21.15.99:0/3133536646 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd70c011a20 con 0x7fd710109080 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 -- 172.21.15.99:0/3133536646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 msgr2=0x7fd7101094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 --2- 172.21.15.99:0/3133536646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd7101094a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098a0 tx=0x7fd70c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:55.791 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 -- 172.21.15.99:0/3133536646 shutdown_connections 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 --2- 172.21.15.99:0/3133536646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd7101094a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 -- 172.21.15.99:0/3133536646 >> 172.21.15.99:0/3133536646 conn(0x7fd710104720 msgr2=0x7fd710106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 -- 172.21.15.99:0/3133536646 shutdown_connections 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.749+0000 7fd717ddc700 1 -- 172.21.15.99:0/3133536646 wait complete. 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd717ddc700 1 Processor -- start 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd717ddc700 1 -- start start 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd717ddc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd71019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd717ddc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010b0c0 con 0x7fd710109080 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd715b78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd71019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd715b78700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd71019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54704/0 (socket says 172.21.15.99:54704) 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd715b78700 1 -- 172.21.15.99:0/3220287307 learned_addr learned my addr 172.21.15.99:0/3220287307 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd715b78700 1 -- 172.21.15.99:0/3220287307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71019d430 con 0x7fd710109080 2024-05-14T16:52:55.792 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.750+0000 7fd715b78700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd71019cef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0116e0 tx=0x7fd70c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.751+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd70c003a60 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.751+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd70c01b8f0 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.751+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71019d6c0 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.751+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd70c019860 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.751+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a03b0 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.752+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd70c019b10 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.752+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd710064ba0 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.752+0000 7fd706ffd700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fd6fc03e470 0x7fd6fc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.752+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd70c053c00 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.752+0000 7fd715377700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fd6fc03e470 0x7fd6fc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.753+0000 7fd715377700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fd6fc03e470 0x7fd6fc040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd700006fd0 tx=0x7fd700006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:54.757+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd70c01f030 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.033+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd7101a0580 con 0x7fd710109080 2024-05-14T16:52:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.201+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd70c028de0 con 0x7fd710109080 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.202+0000 7fd706ffd700 1 -- 172.21.15.99:0/3220287307 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fd70c00fd80 con 0x7fd710109080 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fd6fc03e470 msgr2=0x7fd6fc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fd6fc03e470 0x7fd6fc040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd700006fd0 tx=0x7fd700006e40 comp rx=0 tx=0).stop 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 msgr2=0x7fd71019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd71019cef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0116e0 tx=0x7fd70c011aa0 comp rx=0 tx=0).stop 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 shutdown_connections 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7fd6fc03e470 0x7fd6fc040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 --2- 172.21.15.99:0/3220287307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd710109080 0x7fd71019cef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 >> 172.21.15.99:0/3220287307 conn(0x7fd710104720 msgr2=0x7fd710105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 shutdown_connections 2024-05-14T16:52:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:55.204+0000 7fd717ddc700 1 -- 172.21.15.99:0/3220287307 wait complete. 2024-05-14T16:52:55.921 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:55 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:55.669+0000 7f403b7ff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:52:56.201 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:55 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:55.990+0000 7f403b7ff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:52:56.201 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:56 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:56.081+0000 7f403b7ff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:52:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:56 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3220287307' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-14T16:52:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:56 smithi099 ceph-mon[31851]: mgrmap e9: x(active, since 19s) 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:52:57.098 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15ccaea700 1 Processor -- start 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15ccaea700 1 -- start start 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15ccaea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c810bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15ccaea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15c807ce20 con 0x7f15c8109b30 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c810bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c810bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54738/0 (socket says 172.21.15.99:54738) 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.222+0000 7f15c659c700 1 -- 172.21.15.99:0/265825205 learned_addr learned my addr 172.21.15.99:0/265825205 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.223+0000 7f15c659c700 1 -- 172.21.15.99:0/265825205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c810c460 con 0x7f15c8109b30 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.223+0000 7f15c659c700 1 --2- 172.21.15.99:0/265825205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c810bf20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15b80098a0 tx=0x7f15b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b5ef8eb82f8bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.223+0000 7f15c559a700 1 -- 172.21.15.99:0/265825205 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b8004160 con 0x7f15c8109b30 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.223+0000 7f15c559a700 1 -- 172.21.15.99:0/265825205 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b8004300 con 0x7f15c8109b30 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.223+0000 7f15c559a700 1 -- 172.21.15.99:0/265825205 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b8011a20 con 0x7f15c8109b30 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.223+0000 7f15ccaea700 1 -- 172.21.15.99:0/265825205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 msgr2=0x7f15c810bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 --2- 172.21.15.99:0/265825205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c810bf20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15b80098a0 tx=0x7f15b8009bb0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.099 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 -- 172.21.15.99:0/265825205 shutdown_connections 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 --2- 172.21.15.99:0/265825205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c810bf20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 -- 172.21.15.99:0/265825205 >> 172.21.15.99:0/265825205 conn(0x7f15c8103500 msgr2=0x7f15c8105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 -- 172.21.15.99:0/265825205 shutdown_connections 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 -- 172.21.15.99:0/265825205 wait complete. 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 Processor -- start 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.224+0000 7f15ccaea700 1 -- start start 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15ccaea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c81093b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15ccaea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d400 con 0x7f15c8109b30 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c81093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15c659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c81093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54744/0 (socket says 172.21.15.99:54744) 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15c659c700 1 -- 172.21.15.99:0/2443737231 learned_addr learned my addr 172.21.15.99:0/2443737231 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15c659c700 1 -- 172.21.15.99:0/2443737231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c81098f0 con 0x7f15c8109b30 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15c659c700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c81093b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b8003aa0 con 0x7f15c8109b30 2024-05-14T16:52:57.100 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b801b8f0 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15b8023870 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c8107ae0 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.225+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c8108020 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.226+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f15b8023b20 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.226+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c8064ba0 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.227+0000 7f15b77fe700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 0x7f15b00409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.227+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f15b8053da0 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.227+0000 7f15c5d9b700 1 -- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 msgr2=0x7f15b00409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.227+0000 7f15c5d9b700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 0x7f15b00409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.232+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15b801f030 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.428+0000 7f15c5d9b700 1 -- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 msgr2=0x7f15b00409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.428+0000 7f15c5d9b700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 0x7f15b00409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.535+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f15c81082a0 con 0x7f15c8109b30 2024-05-14T16:52:57.101 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.536+0000 7f15b77fe700 1 -- 172.21.15.99:0/2443737231 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f15b800b3e0 con 0x7f15c8109b30 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 msgr2=0x7f15b00409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 0x7f15b00409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 msgr2=0x7f15c81093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c81093b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b8011aa0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 shutdown_connections 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f15b003e520 0x7f15b00409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 --2- 172.21.15.99:0/2443737231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15c8109b30 0x7f15c81093b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.538+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 >> 172.21.15.99:0/2443737231 conn(0x7f15c8103500 msgr2=0x7f15c8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.539+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 shutdown_connections 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:56.539+0000 7f15ccaea700 1 -- 172.21.15.99:0/2443737231 wait complete. 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-05-14T16:52:57.102 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 9... 2024-05-14T16:52:57.347 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:52:57 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2443737231' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T16:52:57.695 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:57 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:57.347+0000 7f403b7ff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:52:57.695 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:57 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:57.523+0000 7f403b7ff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:52:57.695 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:57 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:57.695+0000 7f403b7ff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:52:58.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:57 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:57.987+0000 7f403b7ff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:52:58.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:58 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:58.080+0000 7f403b7ff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:52:58.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:58 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:58.284+0000 7f403b7ff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:52:58.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:58 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:58.479+0000 7f403b7ff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:52:59.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:59 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:59.081+0000 7f403b7ff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:52:59.733 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:59 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:59.306+0000 7f403b7ff200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:53:00.014 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:59 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:59.733+0000 7f403b7ff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:53:00.014 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:52:59 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:52:59.831+0000 7f403b7ff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:53:00.514 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:00.200+0000 7f403b7ff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:53:00.896 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:00.558+0000 7f403b7ff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:53:00.896 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:00.652+0000 7f403b7ff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:53:01.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:00.896+0000 7f403b7ff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:53:01.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:01 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:01.030+0000 7f403b7ff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:53:01.264 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:01 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:01.127+0000 7f403b7ff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:53:01.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:01 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:01.451+0000 7f403b7ff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:53:01.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:01 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:01.543+0000 7f403b7ff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:53:02.420 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:02 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:02.146+0000 7f403b7ff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:53:02.420 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 16:53:02 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T16:53:02.367+0000 7f403b7ff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:53:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: Active manager daemon x restarted 2024-05-14T16:53:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: Activating manager daemon x 2024-05-14T16:53:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: osdmap e3: 0 total, 0 up, 0 in 2024-05-14T16:53:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: mgrmap e10: x(active, starting, since 0.00346219s) 2024-05-14T16:53:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:53:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:53:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:53:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:53:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:53:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: Manager daemon x is now available 2024-05-14T16:53:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:53:03.575 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:53:03.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:03 smithi099 ceph-mon[31851]: [14/May/2024:16:53:02] ENGINE Bus STARTING 2024-05-14T16:53:03.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:03 smithi099 ceph-mon[31851]: [14/May/2024:16:53:02] ENGINE Serving on http://172.21.15.99:8765 2024-05-14T16:53:03.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:03 smithi099 ceph-mon[31851]: [14/May/2024:16:53:03] ENGINE Serving on https://172.21.15.99:7150 2024-05-14T16:53:03.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:03 smithi099 ceph-mon[31851]: [14/May/2024:16:53:03] ENGINE Bus STARTED 2024-05-14T16:53:03.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:03 smithi099 ceph-mon[31851]: mgrmap e11: x(active, since 1.00692s) 2024-05-14T16:53:04.127 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:53:04.136 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-14T16:53:04.136 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T16:53:04.136 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:53:04.136 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.537+0000 7f30548ea700 1 Processor -- start 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.537+0000 7f30548ea700 1 -- start start 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f30548ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305010bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f30548ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f305010c3e0 con 0x7f3050109ab0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f304e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305010bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f304e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305010bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54756/0 (socket says 172.21.15.99:54756) 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f304e59c700 1 -- 172.21.15.99:0/2138272642 learned_addr learned my addr 172.21.15.99:0/2138272642 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f304e59c700 1 -- 172.21.15.99:0/2138272642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305010c560 con 0x7f3050109ab0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.538+0000 7f304e59c700 1 --2- 172.21.15.99:0/2138272642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305010bea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30440098a0 tx=0x7f3044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdb96d4b5ab0503 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f304d59a700 1 -- 172.21.15.99:0/2138272642 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3044004160 con 0x7f3050109ab0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f304d59a700 1 -- 172.21.15.99:0/2138272642 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3044004300 con 0x7f3050109ab0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f304d59a700 1 -- 172.21.15.99:0/2138272642 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3044011a20 con 0x7f3050109ab0 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 -- 172.21.15.99:0/2138272642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 msgr2=0x7f305010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:04.137 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 --2- 172.21.15.99:0/2138272642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305010bea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30440098a0 tx=0x7f3044009bb0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 -- 172.21.15.99:0/2138272642 shutdown_connections 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 --2- 172.21.15.99:0/2138272642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305010bea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 -- 172.21.15.99:0/2138272642 >> 172.21.15.99:0/2138272642 conn(0x7f30501034e0 msgr2=0x7f3050105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 -- 172.21.15.99:0/2138272642 shutdown_connections 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.539+0000 7f30548ea700 1 -- 172.21.15.99:0/2138272642 wait complete. 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.540+0000 7f30548ea700 1 Processor -- start 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.540+0000 7f30548ea700 1 -- start start 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f30548ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f30548ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f305010d500 con 0x7f3050109ab0 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f304e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f304e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54764/0 (socket says 172.21.15.99:54764) 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f304e59c700 1 -- 172.21.15.99:0/2765049260 learned_addr learned my addr 172.21.15.99:0/2765049260 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:04.138 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f304e59c700 1 -- 172.21.15.99:0/2765049260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305019d560 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f304e59c700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305019d020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3044000c00 tx=0x7f3044011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3044003aa0 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305019d7f0 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304401b8f0 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3044023870 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.541+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30501a04e0 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.542+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f3044023b70 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.543+0000 7f303f7fe700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.543+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 --> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3038041090 con 0x7f303803e4c0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.543+0000 7f304dd9b700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 msgr2=0x7f3038040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.543+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3044053f80 con 0x7f3050109ab0 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.543+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.744+0000 7f304dd9b700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 msgr2=0x7f3038040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:53:04.139 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:57.744+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:58.145+0000 7f304dd9b700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 msgr2=0x7f3038040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:58.145+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:58.946+0000 7f304dd9b700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 msgr2=0x7f3038040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:52:58.946+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:00.547+0000 7f304dd9b700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 msgr2=0x7f3038040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1816772983 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:00.547+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:02.371+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f304402c3f0 con 0x7f3050109ab0 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:02.371+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 msgr2=0x7f3038040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:02.371+0000 7f303f7fe700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.140 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.373+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f3044022360 con 0x7f3050109ab0 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.373+0000 7f303f7fe700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3038041f40 0x7f3038044330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.373+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3044020e70 con 0x7f3038041f40 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.374+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3038041f40 0x7f3038044330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.374+0000 7f304dd9b700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3038041f40 0x7f3038044330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3040003a10 tx=0x7f30400092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.375+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3044020e70 con 0x7f3038041f40 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.383+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f305006a6a0 con 0x7f3038041f40 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.383+0000 7f303f7fe700 1 -- 172.21.15.99:0/2765049260 <== mgr.14152 v2:172.21.15.99:6800/2498052825 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f305006a6a0 con 0x7f3038041f40 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3038041f40 msgr2=0x7f3038044330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3038041f40 0x7f3038044330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3040003a10 tx=0x7f30400092b0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 msgr2=0x7f305019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305019d020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3044000c00 tx=0x7f3044011aa0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 shutdown_connections 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3050109ab0 0x7f305019d020 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.141 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3038041f40 0x7f3038044330 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 --2- 172.21.15.99:0/2765049260 >> [v2:172.21.15.99:6800/1816772983,v1:172.21.15.99:6801/1816772983] conn(0x7f303803e4c0 0x7f3038040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:04.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 >> 172.21.15.99:0/2765049260 conn(0x7f30501034e0 msgr2=0x7f3050104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:04.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 shutdown_connections 2024-05-14T16:53:04.142 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:03.384+0000 7f30548ea700 1 -- 172.21.15.99:0/2765049260 wait complete. 2024-05-14T16:53:04.142 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 9 is available 2024-05-14T16:53:04.142 INFO:teuthology.orchestra.run.smithi099.stdout:Generating a dashboard self-signed certificate... 2024-05-14T16:53:04.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:04 smithi099 ceph-mon[31851]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T16:53:04.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:04 smithi099 ceph-mon[31851]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.553+0000 7fdc72431700 1 Processor -- start 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.553+0000 7fdc72431700 1 -- start start 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.553+0000 7fdc72431700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.553+0000 7fdc72431700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c108870 con 0x7fdc6c107ef0 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.553+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45940/0 (socket says 172.21.15.99:45940) 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6bfff700 1 -- 172.21.15.99:0/1065650857 learned_addr learned my addr 172.21.15.99:0/1065650857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:05.498 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6bfff700 1 -- 172.21.15.99:0/1065650857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c1090d0 con 0x7fdc6c107ef0 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6bfff700 1 --2- 172.21.15.99:0/1065650857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c108330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00bb40 tx=0x7fdc5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24fabc04feef1d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6b7fe700 1 -- 172.21.15.99:0/1065650857 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c00dbe0 con 0x7fdc6c107ef0 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6b7fe700 1 -- 172.21.15.99:0/1065650857 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c0044e0 con 0x7fdc6c107ef0 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.554+0000 7fdc6b7fe700 1 -- 172.21.15.99:0/1065650857 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c010470 con 0x7fdc6c107ef0 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 -- 172.21.15.99:0/1065650857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 msgr2=0x7fdc6c108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 --2- 172.21.15.99:0/1065650857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c108330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00bb40 tx=0x7fdc5c00be50 comp rx=0 tx=0).stop 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 -- 172.21.15.99:0/1065650857 shutdown_connections 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 --2- 172.21.15.99:0/1065650857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c108330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 -- 172.21.15.99:0/1065650857 >> 172.21.15.99:0/1065650857 conn(0x7fdc6c1034c0 msgr2=0x7fdc6c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 -- 172.21.15.99:0/1065650857 shutdown_connections 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.555+0000 7fdc72431700 1 -- 172.21.15.99:0/1065650857 wait complete. 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc72431700 1 Processor -- start 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc72431700 1 -- start start 2024-05-14T16:53:05.499 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc72431700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc72431700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c07bfc0 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45944/0 (socket says 172.21.15.99:45944) 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc6bfff700 1 -- 172.21.15.99:0/1277420036 learned_addr learned my addr 172.21.15.99:0/1277420036 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc6bfff700 1 -- 172.21.15.99:0/1277420036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c19b340 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc6bfff700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c19ae00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00b340 tx=0x7fdc5c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c027070 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c010c20 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c022b70 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.556+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c19b5d0 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.557+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c19e270 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.557+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fdc5c02a070 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.557+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c064ba0 con 0x7fdc6c107ef0 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.558+0000 7fdc69ffb700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdc5803e470 0x7fdc58040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.558+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc5c01f070 con 0x7fdc6c107ef0 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.558+0000 7fdc63fff700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdc5803e470 0x7fdc58040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.558+0000 7fdc63fff700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdc5803e470 0x7fdc58040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc54006fd0 tx=0x7fdc54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.563+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc5c015070 con 0x7fdc6c107ef0 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:04.792+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fdc6c063980 con 0x7fdc5803e470 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.010+0000 7fdc69ffb700 1 -- 172.21.15.99:0/1277420036 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fdc6c063980 con 0x7fdc5803e470 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdc5803e470 msgr2=0x7fdc58040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdc5803e470 0x7fdc58040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc54006fd0 tx=0x7fdc54006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 msgr2=0x7fdc6c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c19ae00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00b340 tx=0x7fdc5c003dc0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 shutdown_connections 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdc5803e470 0x7fdc58040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 --2- 172.21.15.99:0/1277420036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc6c107ef0 0x7fdc6c19ae00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 >> 172.21.15.99:0/1277420036 conn(0x7fdc6c1034c0 msgr2=0x7fdc6c191380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:05.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 shutdown_connections 2024-05-14T16:53:05.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.012+0000 7fdc72431700 1 -- 172.21.15.99:0/1277420036 wait complete. 2024-05-14T16:53:05.502 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial admin user... 2024-05-14T16:53:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:05 smithi099 ceph-mon[31851]: mgrmap e12: x(active, since 2s) 2024-05-14T16:53:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:05.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:05 smithi099 ceph-mon[31851]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:05.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:05.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:06.706 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:06 smithi099 ceph-mon[31851]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:06.706 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:06 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:06.951 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vEUg4gX2SRgQhfYQbIkYE.KRHJokiHUNUNIkVSKe7fOgA8M5FISOy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715705586, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-14T16:53:06.951 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.922+0000 7f9797cf7700 1 Processor -- start 2024-05-14T16:53:06.951 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9797cf7700 1 -- start start 2024-05-14T16:53:06.952 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9797cf7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f97901094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:06.952 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9797cf7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9790109a20 con 0x7f97901090c0 2024-05-14T16:53:06.952 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9795a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f97901094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9795a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f97901094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45950/0 (socket says 172.21.15.99:45950) 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9795a93700 1 -- 172.21.15.99:0/2065343598 learned_addr learned my addr 172.21.15.99:0/2065343598 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9795a93700 1 -- 172.21.15.99:0/2065343598 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9790109ba0 con 0x7f97901090c0 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.923+0000 7f9795a93700 1 --2- 172.21.15.99:0/2065343598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f97901094e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff32d930c9db781 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9794a91700 1 -- 172.21.15.99:0/2065343598 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c004160 con 0x7f97901090c0 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9794a91700 1 -- 172.21.15.99:0/2065343598 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978c004300 con 0x7f97901090c0 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9794a91700 1 -- 172.21.15.99:0/2065343598 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c011a20 con 0x7f97901090c0 2024-05-14T16:53:06.953 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 -- 172.21.15.99:0/2065343598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 msgr2=0x7f97901094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:06.954 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 --2- 172.21.15.99:0/2065343598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f97901094e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.954 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 -- 172.21.15.99:0/2065343598 shutdown_connections 2024-05-14T16:53:06.954 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 --2- 172.21.15.99:0/2065343598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f97901094e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.954 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 -- 172.21.15.99:0/2065343598 >> 172.21.15.99:0/2065343598 conn(0x7f9790104720 msgr2=0x7f9790106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:06.954 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 -- 172.21.15.99:0/2065343598 shutdown_connections 2024-05-14T16:53:06.954 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.924+0000 7f9797cf7700 1 -- 172.21.15.99:0/2065343598 wait complete. 2024-05-14T16:53:06.955 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9797cf7700 1 Processor -- start 2024-05-14T16:53:06.955 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9797cf7700 1 -- start start 2024-05-14T16:53:06.955 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9797cf7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f979019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:06.955 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9797cf7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f979010b0b0 con 0x7f97901090c0 2024-05-14T16:53:06.956 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9795a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f979019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:06.956 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9795a93700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f979019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45956/0 (socket says 172.21.15.99:45956) 2024-05-14T16:53:06.956 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9795a93700 1 -- 172.21.15.99:0/891351918 learned_addr learned my addr 172.21.15.99:0/891351918 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:06.956 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.925+0000 7f9795a93700 1 -- 172.21.15.99:0/891351918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979019d460 con 0x7f97901090c0 2024-05-14T16:53:06.956 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.926+0000 7f9795a93700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f979019cf20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f978c0116e0 tx=0x7f978c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:06.956 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.926+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c003a60 con 0x7f97901090c0 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.926+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978c01b8f0 con 0x7f97901090c0 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.926+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979019d6f0 con 0x7f97901090c0 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.926+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c019860 con 0x7f97901090c0 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.926+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97901a03e0 con 0x7f97901090c0 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.927+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f978c019b10 con 0x7f97901090c0 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.927+0000 7f9786ffd700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f977c03e4d0 0x7f977c040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:06.957 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.927+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f978c053d50 con 0x7f97901090c0 2024-05-14T16:53:06.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.927+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9790064ba0 con 0x7f97901090c0 2024-05-14T16:53:06.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.927+0000 7f9795292700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f977c03e4d0 0x7f977c040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:06.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.927+0000 7f9795292700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f977c03e4d0 0x7f977c040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9780006fd0 tx=0x7f9780006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:06.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:05.932+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f978c01f030 con 0x7f97901090c0 2024-05-14T16:53:06.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.164+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f979019dbd0 con 0x7f977c03e4d0 2024-05-14T16:53:06.958 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.387+0000 7f9786ffd700 1 -- 172.21.15.99:0/891351918 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f979019dbd0 con 0x7f977c03e4d0 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f977c03e4d0 msgr2=0x7f977c040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f977c03e4d0 0x7f977c040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9780006fd0 tx=0x7f9780006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 msgr2=0x7f979019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f979019cf20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f978c0116e0 tx=0x7f978c011aa0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 shutdown_connections 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f977c03e4d0 0x7f977c040990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.389+0000 7f9797cf7700 1 --2- 172.21.15.99:0/891351918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97901090c0 0x7f979019cf20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.959 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.390+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 >> 172.21.15.99:0/891351918 conn(0x7f9790104720 msgr2=0x7f97901051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:06.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.390+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 shutdown_connections 2024-05-14T16:53:06.960 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:06.390+0000 7f9797cf7700 1 -- 172.21.15.99:0/891351918 wait complete. 2024-05-14T16:53:06.960 INFO:teuthology.orchestra.run.smithi099.stdout:Fetching dashboard port number... 2024-05-14T16:53:07.635 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 8443 2024-05-14T16:53:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f054a342700 1 Processor -- start 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f054a342700 1 -- start start 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f054a342700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054410bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f054a342700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f054410c400 con 0x7f0544109ad0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f0543fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054410bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f0543fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054410bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45958/0 (socket says 172.21.15.99:45958) 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f0543fff700 1 -- 172.21.15.99:0/2296055369 learned_addr learned my addr 172.21.15.99:0/2296055369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f0543fff700 1 -- 172.21.15.99:0/2296055369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054410c580 con 0x7f0544109ad0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.386+0000 7f0543fff700 1 --2- 172.21.15.99:0/2296055369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054410bec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05340098a0 tx=0x7f0534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f10cdb55421218c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f0542ffd700 1 -- 172.21.15.99:0/2296055369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0534004160 con 0x7f0544109ad0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f0542ffd700 1 -- 172.21.15.99:0/2296055369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0534004300 con 0x7f0544109ad0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f0542ffd700 1 -- 172.21.15.99:0/2296055369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0534011a20 con 0x7f0544109ad0 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f054a342700 1 -- 172.21.15.99:0/2296055369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 msgr2=0x7f054410bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f054a342700 1 --2- 172.21.15.99:0/2296055369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054410bec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05340098a0 tx=0x7f0534009bb0 comp rx=0 tx=0).stop 2024-05-14T16:53:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f054a342700 1 -- 172.21.15.99:0/2296055369 shutdown_connections 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f054a342700 1 --2- 172.21.15.99:0/2296055369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054410bec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.387+0000 7f054a342700 1 -- 172.21.15.99:0/2296055369 >> 172.21.15.99:0/2296055369 conn(0x7f0544103500 msgr2=0x7f0544105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.388+0000 7f054a342700 1 -- 172.21.15.99:0/2296055369 shutdown_connections 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.388+0000 7f054a342700 1 -- 172.21.15.99:0/2296055369 wait complete. 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.388+0000 7f054a342700 1 Processor -- start 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.388+0000 7f054a342700 1 -- start start 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f054a342700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f054a342700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f054410d520 con 0x7f0544109ad0 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f0543fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f0543fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45964/0 (socket says 172.21.15.99:45964) 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f0543fff700 1 -- 172.21.15.99:0/3444703235 learned_addr learned my addr 172.21.15.99:0/3444703235 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f0543fff700 1 -- 172.21.15.99:0/3444703235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054419d580 con 0x7f0544109ad0 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f0543fff700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054419d040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05340116e0 tx=0x7f0534011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:08.502 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0534003a60 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.389+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054419d810 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.390+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f053401b8f0 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.390+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0534019860 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.390+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05441a0500 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.390+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f0534019b60 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.391+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0544064ba0 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.391+0000 7f05417fa700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f052c03e470 0x7f052c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.391+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0534053de0 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.391+0000 7f05437fe700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f052c03e470 0x7f052c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.391+0000 7f05437fe700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f052c03e470 0x7f052c040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0538006fd0 tx=0x7f0538006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.396+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f053401f030 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.644+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f05441a06d0 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.645+0000 7f05417fa700 1 -- 172.21.15.99:0/3444703235 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f05340223b0 con 0x7f0544109ad0 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f052c03e470 msgr2=0x7f052c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:08.503 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f052c03e470 0x7f052c040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0538006fd0 tx=0x7f0538006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 msgr2=0x7f054419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054419d040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05340116e0 tx=0x7f0534011aa0 comp rx=0 tx=0).stop 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 shutdown_connections 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f052c03e470 0x7f052c040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 --2- 172.21.15.99:0/3444703235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0544109ad0 0x7f054419d040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 >> 172.21.15.99:0/3444703235 conn(0x7f0544103500 msgr2=0x7f0544103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 shutdown_connections 2024-05-14T16:53:08.504 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:07.647+0000 7f054a342700 1 -- 172.21.15.99:0/3444703235 wait complete. 2024-05-14T16:53:08.507 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:53:08.507 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-05-14T16:53:08.515 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:53:08.515 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-05-14T16:53:08.515 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph Dashboard is now available at: 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout: URL: https://smithi099.front.sepia.ceph.com:8443/ 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout: User: admin 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout: Password: 0m82uaqj1m 2024-05-14T16:53:08.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:08.517 INFO:teuthology.orchestra.run.smithi099.stdout:Saving cluster configuration to /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config directory 2024-05-14T16:53:08.517 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling autotune for osd_memory_target 2024-05-14T16:53:08.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:08 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3444703235' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-14T16:53:09.801 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:09 smithi099 ceph-mon[31851]: mgrmap e13: x(active, since 6s) 2024-05-14T16:53:10.085 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.308+0000 7f3ffc162700 1 Processor -- start 2024-05-14T16:53:10.085 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.308+0000 7f3ffc162700 1 -- start start 2024-05-14T16:53:10.085 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.308+0000 7f3ffc162700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff4107d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:10.085 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.308+0000 7f3ffc162700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4076e00 con 0x7f3ff4107900 2024-05-14T16:53:10.085 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff9efe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff4107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff9efe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff4107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45968/0 (socket says 172.21.15.99:45968) 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff9efe700 1 -- 172.21.15.99:0/1519572082 learned_addr learned my addr 172.21.15.99:0/1519572082 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff9efe700 1 -- 172.21.15.99:0/1519572082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff4108260 con 0x7f3ff4107900 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff9efe700 1 --2- 172.21.15.99:0/1519572082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff4107d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00098a0 tx=0x7f3ff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77264b3909bbfbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff8efc700 1 -- 172.21.15.99:0/1519572082 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0004160 con 0x7f3ff4107900 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff8efc700 1 -- 172.21.15.99:0/1519572082 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff0004300 con 0x7f3ff4107900 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.309+0000 7f3ff8efc700 1 -- 172.21.15.99:0/1519572082 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0011a20 con 0x7f3ff4107900 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 -- 172.21.15.99:0/1519572082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 msgr2=0x7f3ff4107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 --2- 172.21.15.99:0/1519572082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff4107d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00098a0 tx=0x7f3ff0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 -- 172.21.15.99:0/1519572082 shutdown_connections 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 --2- 172.21.15.99:0/1519572082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff4107d20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 -- 172.21.15.99:0/1519572082 >> 172.21.15.99:0/1519572082 conn(0x7f3ff41034e0 msgr2=0x7f3ff4105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:10.086 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 -- 172.21.15.99:0/1519572082 shutdown_connections 2024-05-14T16:53:10.087 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.310+0000 7f3ffc162700 1 -- 172.21.15.99:0/1519572082 wait complete. 2024-05-14T16:53:10.087 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ffc162700 1 Processor -- start 2024-05-14T16:53:10.087 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ffc162700 1 -- start start 2024-05-14T16:53:10.087 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ffc162700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:10.087 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ffc162700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41092d0 con 0x7f3ff4107900 2024-05-14T16:53:10.088 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ff9efe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:10.088 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ff9efe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45976/0 (socket says 172.21.15.99:45976) 2024-05-14T16:53:10.088 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ff9efe700 1 -- 172.21.15.99:0/958696000 learned_addr learned my addr 172.21.15.99:0/958696000 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:10.088 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ff9efe700 1 -- 172.21.15.99:0/958696000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff419f6d0 con 0x7f3ff4107900 2024-05-14T16:53:10.088 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3ff9efe700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff419f190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00116e0 tx=0x7f3ff0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:10.088 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.311+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0003b20 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.312+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff419f960 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.312+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff001b8f0 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.312+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff00238c0 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.312+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41a2650 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.313+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff4064ba0 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.313+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3ff001baa0 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.313+0000 7f3feaffd700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3fe003e510 0x7f3fe00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.313+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ff00541b0 con 0x7f3ff4107900 2024-05-14T16:53:10.089 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.313+0000 7f3ff96fd700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3fe003e510 0x7f3fe00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:10.090 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.313+0000 7f3ff96fd700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3fe003e510 0x7f3fe00409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4006fd0 tx=0x7f3fe4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:10.090 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.318+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ff001f030 con 0x7f3ff4107900 2024-05-14T16:53:10.090 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.520+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f3ff402e860 con 0x7f3ff4107900 2024-05-14T16:53:10.090 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.520+0000 7f3feaffd700 1 -- 172.21.15.99:0/958696000 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f3ff0018590 con 0x7f3ff4107900 2024-05-14T16:53:10.090 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3fe003e510 msgr2=0x7f3fe00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:10.091 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3fe003e510 0x7f3fe00409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4006fd0 tx=0x7f3fe4006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:10.091 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 msgr2=0x7f3ff419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:10.091 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff419f190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00116e0 tx=0x7f3ff0011aa0 comp rx=0 tx=0).stop 2024-05-14T16:53:10.091 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 shutdown_connections 2024-05-14T16:53:10.091 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3fe003e510 0x7f3fe00409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:10.091 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 --2- 172.21.15.99:0/958696000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4107900 0x7f3ff419f190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:10.092 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 >> 172.21.15.99:0/958696000 conn(0x7f3ff41034e0 msgr2=0x7f3ff4195db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:10.092 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 shutdown_connections 2024-05-14T16:53:10.092 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:09.522+0000 7f3ffc162700 1 -- 172.21.15.99:0/958696000 wait complete. 2024-05-14T16:53:11.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb73e570700 1 Processor -- start 2024-05-14T16:53:11.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb73e570700 1 -- start start 2024-05-14T16:53:11.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb73e570700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73810bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb73e570700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb73807ce40 con 0x7fb738109b60 2024-05-14T16:53:11.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73810bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73810bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45982/0 (socket says 172.21.15.99:45982) 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.456+0000 7fb737fff700 1 -- 172.21.15.99:0/2437199411 learned_addr learned my addr 172.21.15.99:0/2437199411 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb737fff700 1 -- 172.21.15.99:0/2437199411 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73810c490 con 0x7fb738109b60 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb737fff700 1 --2- 172.21.15.99:0/2437199411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73810bf50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098a0 tx=0x7fb728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e695e80cdd1737c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb736ffd700 1 -- 172.21.15.99:0/2437199411 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb728004160 con 0x7fb738109b60 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb736ffd700 1 -- 172.21.15.99:0/2437199411 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb728004300 con 0x7fb738109b60 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb736ffd700 1 -- 172.21.15.99:0/2437199411 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb728011a20 con 0x7fb738109b60 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb73e570700 1 -- 172.21.15.99:0/2437199411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 msgr2=0x7fb73810bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.457+0000 7fb73e570700 1 --2- 172.21.15.99:0/2437199411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73810bf50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7280098a0 tx=0x7fb728009bb0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 -- 172.21.15.99:0/2437199411 shutdown_connections 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 --2- 172.21.15.99:0/2437199411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73810bf50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 -- 172.21.15.99:0/2437199411 >> 172.21.15.99:0/2437199411 conn(0x7fb738103570 msgr2=0x7fb738105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 -- 172.21.15.99:0/2437199411 shutdown_connections 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 -- 172.21.15.99:0/2437199411 wait complete. 2024-05-14T16:53:11.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 Processor -- start 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.458+0000 7fb73e570700 1 -- start start 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb73e570700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb73e570700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb73810d440 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb737fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45992/0 (socket says 172.21.15.99:45992) 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb737fff700 1 -- 172.21.15.99:0/1055843258 learned_addr learned my addr 172.21.15.99:0/1055843258 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb737fff700 1 -- 172.21.15.99:0/1055843258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73819d4d0 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb737fff700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73819cf90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb728004660 tx=0x7fb728011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb728003aa0 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb72801b8f0 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73819d760 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb728023900 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.459+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7381a0450 con 0x7fb738109b60 2024-05-14T16:53:11.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.460+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb728023b70 con 0x7fb738109b60 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.460+0000 7fb7357fa700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb72003e5c0 0x7fb720040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.460+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73819dc40 con 0x7fb738109b60 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.460+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb728054000 con 0x7fb738109b60 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.460+0000 7fb7377fe700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb72003e5c0 0x7fb720040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.461+0000 7fb7377fe700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb72003e5c0 0x7fb720040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb72c006fd0 tx=0x7fb72c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.465+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb73819dc40 con 0x7fb738109b60 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.782+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb7381a0620 con 0x7fb738109b60 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.784+0000 7fb7357fa700 1 -- 172.21.15.99:0/1055843258 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fb72801f030 con 0x7fb738109b60 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb72003e5c0 msgr2=0x7fb720040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb72003e5c0 0x7fb720040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb72c006fd0 tx=0x7fb72c006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 msgr2=0x7fb73819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73819cf90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb728004660 tx=0x7fb728011aa0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 shutdown_connections 2024-05-14T16:53:11.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb72003e5c0 0x7fb720040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 --2- 172.21.15.99:0/1055843258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb738109b60 0x7fb73819cf90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 >> 172.21.15.99:0/1055843258 conn(0x7fb738103570 msgr2=0x7fb738104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 shutdown_connections 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-05-14T16:53:10.787+0000 7fb73e570700 1 -- 172.21.15.99:0/1055843258 wait complete. 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:Or, if you are only running a single cluster on this host: 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.355 INFO:teuthology.orchestra.run.smithi099.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: ceph telemetry on 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout:For more information see: 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:11.356 INFO:teuthology.orchestra.run.smithi099.stdout:Bootstrap complete. 2024-05-14T16:53:11.390 INFO:tasks.cephadm:Fetching config... 2024-05-14T16:53:11.390 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:53:11.390 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-14T16:53:11.407 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-14T16:53:11.408 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:53:11.408 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-14T16:53:11.463 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-14T16:53:11.463 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:53:11.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-14T16:53:11.528 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-14T16:53:11.529 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:53:11.529 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-14T16:53:11.583 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-14T16:53:11.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:53:11.657 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:11.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:53:11.706 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:11.718 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:53:11.757 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcnPhX1ucgwgMUwI752y7TJUNAfDUH2RMHsFHlNRusqcYFSYF1YU15uCsVgxrUMnnN+740HnhO5BzTGmDGCrVbKmDwLkHYHR/BO7sjUslmjMmx17nDF+e2hwwvOZ8o8y7l1C1S0Ti9+gJ4eKy1UK/ZQTUyo2m66P6/c5CUqVruknSd4Lcy75BPzTwn9a/T6clf0stJcV8fe3qj5SiFCKJeflOHtsIWmhUVylA6GkmJMTtjF+7Rz/3UkDw06Kg/wCGI4uxSw8xYGSbiIEnksAb6Am2Jl1OkrLZRV2DuWuajJy0lSWx1VJ2lCHKZa311WKhlfwm/tGC+Q+sP/PmsocJuJCangPz5gSTrtToVxPagotyF3Am9VGnD4TD1vYgK3JTjRzEKhVZ7GKaB+DyC2dgAP9vUbTl1+gV69NykqkAyXxPzyj+feO8W6C2R3b5Eo469kG2QW1uRWkaeVWgk1ahsso/62s81meNuGoCAxp6YoAZQAUGoVubuS6fSroPbZ9E= ceph-2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:11.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-14T16:53:12.010 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:11 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1055843258' entity='client.admin' 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 -- 172.21.15.99:0/2463488379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 msgr2=0x7f7cec102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 --2- 172.21.15.99:0/2463488379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec102270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009910 tx=0x7f7ce0009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 -- 172.21.15.99:0/2463488379 shutdown_connections 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 --2- 172.21.15.99:0/2463488379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec102270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 -- 172.21.15.99:0/2463488379 >> 172.21.15.99:0/2463488379 conn(0x7f7cec0fd440 msgr2=0x7f7cec0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 -- 172.21.15.99:0/2463488379 shutdown_connections 2024-05-14T16:53:14.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.091+0000 7f7cf2a15700 1 -- 172.21.15.99:0/2463488379 wait complete. 2024-05-14T16:53:14.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cf2a15700 1 Processor -- start 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cf2a15700 1 -- start start 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cf2a15700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cf2a15700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec107d50 con 0x7f7cec101e50 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46012/0 (socket says 172.21.15.99:46012) 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cebfff700 1 -- 172.21.15.99:0/8450985 learned_addr learned my addr 172.21.15.99:0/8450985 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.092+0000 7f7cebfff700 1 -- 172.21.15.99:0/8450985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec198b10 con 0x7f7cec101e50 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.093+0000 7f7cebfff700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec1985d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005950 tx=0x7f7ce0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.093+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ce0026070 con 0x7f7cec101e50 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.093+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec198da0 con 0x7f7cec101e50 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.093+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce000bc00 con 0x7f7cec101e50 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.093+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ce0003a10 con 0x7f7cec101e50 2024-05-14T16:53:14.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.093+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec19ba90 con 0x7f7cec101e50 2024-05-14T16:53:14.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.094+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7ce0029070 con 0x7f7cec101e50 2024-05-14T16:53:14.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.094+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec1084d0 con 0x7f7cec101e50 2024-05-14T16:53:14.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.094+0000 7f7ce97fa700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7cd403e510 0x7f7cd40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:14.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.094+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ce001e070 con 0x7f7cec101e50 2024-05-14T16:53:14.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.094+0000 7f7ceb7fe700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7cd403e510 0x7f7cd40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:14.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.094+0000 7f7ceb7fe700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7cd403e510 0x7f7cd40409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc006fd0 tx=0x7f7cdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:14.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.099+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ce0013070 con 0x7f7cec101e50 2024-05-14T16:53:14.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.307+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f7cec02e670 con 0x7f7cec101e50 2024-05-14T16:53:14.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.310+0000 7f7ce97fa700 1 -- 172.21.15.99:0/8450985 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f7ce0030020 con 0x7f7cec101e50 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7cd403e510 msgr2=0x7f7cd40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7cd403e510 0x7f7cd40409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc006fd0 tx=0x7f7cdc006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 msgr2=0x7f7cec1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec1985d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005950 tx=0x7f7ce0004fa0 comp rx=0 tx=0).stop 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 shutdown_connections 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7cd403e510 0x7f7cd40409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 --2- 172.21.15.99:0/8450985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cec101e50 0x7f7cec1985d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 >> 172.21.15.99:0/8450985 conn(0x7f7cec0fd440 msgr2=0x7f7cec06adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 shutdown_connections 2024-05-14T16:53:14.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:14.312+0000 7f7cf2a15700 1 -- 172.21.15.99:0/8450985 wait complete. 2024-05-14T16:53:14.895 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-14T16:53:14.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/8450985' entity='client.admin' 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:14.920 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:15.129 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:16.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 -- 172.21.15.99:0/2829718758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 msgr2=0x7f0fe41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 --2- 172.21.15.99:0/2829718758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe41049f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009910 tx=0x7f0fd4009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 -- 172.21.15.99:0/2829718758 shutdown_connections 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 --2- 172.21.15.99:0/2829718758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe41049f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 -- 172.21.15.99:0/2829718758 >> 172.21.15.99:0/2829718758 conn(0x7f0fe40fcbf0 msgr2=0x7f0fe40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 -- 172.21.15.99:0/2829718758 shutdown_connections 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.607+0000 7f0fe93c3700 1 -- 172.21.15.99:0/2829718758 wait complete. 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe93c3700 1 Processor -- start 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe93c3700 1 -- start start 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe93c3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe93c3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4108e90 con 0x7f0fe41045d0 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe4198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60244/0 (socket says 172.21.15.99:60244) 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe2ffd700 1 -- 172.21.15.99:0/1129231571 learned_addr learned my addr 172.21.15.99:0/1129231571 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.608+0000 7f0fe2ffd700 1 -- 172.21.15.99:0/1129231571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe4198ad0 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.609+0000 7f0fe2ffd700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe4198590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4005800 tx=0x7f0fd4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.609+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fd4027070 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.609+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fd400be10 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.609+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fd400f460 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.609+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe4198dc0 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.609+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe419ba50 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.610+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0fd402a070 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.610+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe4109610 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.610+0000 7f0fe0ff9700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0fd003e510 0x7f0fd00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.610+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0fd401e030 con 0x7f0fe41045d0 2024-05-14T16:53:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.610+0000 7f0fda7fc700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0fd003e510 0x7f0fd00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:16.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.610+0000 7f0fda7fc700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0fd003e510 0x7f0fd00409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:16.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.615+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fd4013070 con 0x7f0fe41045d0 2024-05-14T16:53:16.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.823+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0fe402e6e0 con 0x7f0fd003e510 2024-05-14T16:53:16.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.827+0000 7f0fe0ff9700 1 -- 172.21.15.99:0/1129231571 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0fe402e6e0 con 0x7f0fd003e510 2024-05-14T16:53:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0fd003e510 msgr2=0x7f0fd00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0fd003e510 0x7f0fd00409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 msgr2=0x7f0fe4198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe4198590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4005800 tx=0x7f0fd4009f60 comp rx=0 tx=0).stop 2024-05-14T16:53:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 shutdown_connections 2024-05-14T16:53:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0fd003e510 0x7f0fd00409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 --2- 172.21.15.99:0/1129231571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fe41045d0 0x7f0fe4198590 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 >> 172.21.15.99:0/1129231571 conn(0x7f0fe40fcbf0 msgr2=0x7f0fe40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 shutdown_connections 2024-05-14T16:53:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:16.829+0000 7f0fe93c3700 1 -- 172.21.15.99:0/1129231571 wait complete. 2024-05-14T16:53:17.494 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-05-14T16:53:17.494 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:53:17.494 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-05-14T16:53:17.513 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:53:17.513 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:17.571 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-05-14T16:53:17.571 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch host add smithi150 2024-05-14T16:53:17.834 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:18.152 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:18.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:18.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:18 smithi099 ceph-mon[31851]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:18.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:18 smithi099 ceph-mon[31851]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:53:19.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.168+0000 7f1968b49700 1 -- 172.21.15.99:0/3546977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 msgr2=0x7f19641020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:19.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.168+0000 7f1968b49700 1 --2- 172.21.15.99:0/3546977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641020e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1954009910 tx=0x7f1954009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:19.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.169+0000 7f1968b49700 1 -- 172.21.15.99:0/3546977011 shutdown_connections 2024-05-14T16:53:19.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.169+0000 7f1968b49700 1 --2- 172.21.15.99:0/3546977011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641020e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:19.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.169+0000 7f1968b49700 1 -- 172.21.15.99:0/3546977011 >> 172.21.15.99:0/3546977011 conn(0x7f19640fd920 msgr2=0x7f19640ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.169+0000 7f1968b49700 1 -- 172.21.15.99:0/3546977011 shutdown_connections 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.169+0000 7f1968b49700 1 -- 172.21.15.99:0/3546977011 wait complete. 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f1968b49700 1 Processor -- start 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f1968b49700 1 -- start start 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f1968b49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f1968b49700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f196410f670 con 0x7f1964101d00 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f196259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f196259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60262/0 (socket says 172.21.15.99:60262) 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f196259c700 1 -- 172.21.15.99:0/1939977408 learned_addr learned my addr 172.21.15.99:0/1939977408 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f196259c700 1 -- 172.21.15.99:0/1939977408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19641a0db0 con 0x7f1964101d00 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f196259c700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641a0870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19540056d0 tx=0x7f19540057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.170+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954026070 con 0x7f1964101d00 2024-05-14T16:53:19.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.171+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195400bc00 con 0x7f1964101d00 2024-05-14T16:53:19.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.171+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19641a1040 con 0x7f1964101d00 2024-05-14T16:53:19.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.171+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954003920 con 0x7f1964101d00 2024-05-14T16:53:19.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.171+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196419aa30 con 0x7f1964101d00 2024-05-14T16:53:19.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.171+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196406bc50 con 0x7f1964101d00 2024-05-14T16:53:19.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.171+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1954029070 con 0x7f1964101d00 2024-05-14T16:53:19.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.172+0000 7f19537fe700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f194c03e570 0x7f194c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:19.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.172+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f195401e070 con 0x7f1964101d00 2024-05-14T16:53:19.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.172+0000 7f1961d9b700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f194c03e570 0x7f194c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:19.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.172+0000 7f1961d9b700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f194c03e570 0x7f194c040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:19.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.177+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1954013070 con 0x7f1964101d00 2024-05-14T16:53:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:19.391+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7f196419bf60 con 0x7f194c03e570 2024-05-14T16:53:20.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:20 smithi099 ceph-mon[31851]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:20.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:20 smithi099 ceph-mon[31851]: Deploying cephadm binary to smithi150 2024-05-14T16:53:23.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.705+0000 7f19537fe700 1 -- 172.21.15.99:0/1939977408 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f196419bf60 con 0x7f194c03e570 2024-05-14T16:53:23.705 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-05-14T16:53:23.705 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:23 smithi099 ceph-mon[31851]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f194c03e570 msgr2=0x7f194c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f194c03e570 0x7f194c040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 msgr2=0x7f19641a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641a0870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19540056d0 tx=0x7f19540057b0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 shutdown_connections 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f194c03e570 0x7f194c040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 --2- 172.21.15.99:0/1939977408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1964101d00 0x7f19641a0870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 >> 172.21.15.99:0/1939977408 conn(0x7f19640fd920 msgr2=0x7f19640fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 shutdown_connections 2024-05-14T16:53:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:23.707+0000 7f1968b49700 1 -- 172.21.15.99:0/1939977408 wait complete. 2024-05-14T16:53:24.303 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T16:53:24.555 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:24.931 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:24.932 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:24 smithi099 ceph-mon[31851]: Added host smithi150 2024-05-14T16:53:24.932 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:24.932 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:25.947 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:25 smithi099 ceph-mon[31851]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:25.947 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 -- 172.21.15.99:0/3185876308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 msgr2=0x7f94141024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 --2- 172.21.15.99:0/3185876308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f94141024d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009910 tx=0x7f93fc009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 -- 172.21.15.99:0/3185876308 shutdown_connections 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 --2- 172.21.15.99:0/3185876308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f94141024d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 -- 172.21.15.99:0/3185876308 >> 172.21.15.99:0/3185876308 conn(0x7f94140779d0 msgr2=0x7f9414077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 -- 172.21.15.99:0/3185876308 shutdown_connections 2024-05-14T16:53:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.990+0000 7f9419414700 1 -- 172.21.15.99:0/3185876308 wait complete. 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9419414700 1 Processor -- start 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9419414700 1 -- start start 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9419414700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f941419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9419414700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f941410c350 con 0x7f94141020f0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9412ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f941419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9412ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f941419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34964/0 (socket says 172.21.15.99:34964) 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.991+0000 7f9412ffd700 1 -- 172.21.15.99:0/289953977 learned_addr learned my addr 172.21.15.99:0/289953977 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f9412ffd700 1 -- 172.21.15.99:0/289953977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941419ebc0 con 0x7f94141020f0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f9412ffd700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f941419e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0056d0 tx=0x7f93fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93fc026070 con 0x7f94141020f0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93fc00bba0 con 0x7f94141020f0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93fc003920 con 0x7f94141020f0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941419eeb0 con 0x7f94141020f0 2024-05-14T16:53:25.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.992+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9414198800 con 0x7f94141020f0 2024-05-14T16:53:25.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.993+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f93fc029070 con 0x7f94141020f0 2024-05-14T16:53:25.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.993+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94141035b0 con 0x7f94141020f0 2024-05-14T16:53:25.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.993+0000 7f940bfff700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f940003e510 0x7f94000409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:25.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.993+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93fc01e070 con 0x7f94141020f0 2024-05-14T16:53:25.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.993+0000 7f94127fc700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f940003e510 0x7f94000409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:25.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.994+0000 7f94127fc700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f940003e510 0x7f94000409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9404006fd0 tx=0x7f9404006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:25.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:25.998+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93fc013070 con 0x7f94141020f0 2024-05-14T16:53:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.204+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9414199d30 con 0x7f940003e510 2024-05-14T16:53:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.206+0000 7f940bfff700 1 -- 172.21.15.99:0/289953977 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f9414199d30 con 0x7f940003e510 2024-05-14T16:53:26.205 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:26.205 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-05-14T16:53:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f940003e510 msgr2=0x7f94000409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f940003e510 0x7f94000409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9404006fd0 tx=0x7f9404006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 msgr2=0x7f941419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f941419e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0056d0 tx=0x7f93fc0057b0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 shutdown_connections 2024-05-14T16:53:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f940003e510 0x7f94000409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 --2- 172.21.15.99:0/289953977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94141020f0 0x7f941419e680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 >> 172.21.15.99:0/289953977 conn(0x7f94140779d0 msgr2=0x7f941406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 shutdown_connections 2024-05-14T16:53:26.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:26.209+0000 7f9419414700 1 -- 172.21.15.99:0/289953977 wait complete. 2024-05-14T16:53:26.811 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-05-14T16:53:26.811 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:53:26.811 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-05-14T16:53:26.831 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:53:26.831 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:26.888 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-05-14T16:53:26.888 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch host add smithi193 2024-05-14T16:53:27.121 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:27.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:26 smithi099 ceph-mon[31851]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:27.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:28.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:27 smithi099 ceph-mon[31851]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 -- 172.21.15.99:0/1693082946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 msgr2=0x7f41ac102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 --2- 172.21.15.99:0/1693082946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac102490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f419c00b0a0 tx=0x7f419c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 -- 172.21.15.99:0/1693082946 shutdown_connections 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 --2- 172.21.15.99:0/1693082946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac102490 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 -- 172.21.15.99:0/1693082946 >> 172.21.15.99:0/1693082946 conn(0x7f41ac0fd960 msgr2=0x7f41ac0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 -- 172.21.15.99:0/1693082946 shutdown_connections 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 -- 172.21.15.99:0/1693082946 wait complete. 2024-05-14T16:53:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 Processor -- start 2024-05-14T16:53:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.416+0000 7f41b1484700 1 -- start start 2024-05-14T16:53:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41b1484700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41b1484700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c330 con 0x7f41ac1020b0 2024-05-14T16:53:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34990/0 (socket says 172.21.15.99:34990) 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41aaffd700 1 -- 172.21.15.99:0/819510074 learned_addr learned my addr 172.21.15.99:0/819510074 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41aaffd700 1 -- 172.21.15.99:0/819510074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac199ce0 con 0x7f41ac1020b0 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41aaffd700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac1997a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f419c006130 tx=0x7f419c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f419c021020 con 0x7f41ac1020b0 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac199f70 con 0x7f41ac1020b0 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c0079b0 con 0x7f41ac1020b0 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f419c017040 con 0x7f41ac1020b0 2024-05-14T16:53:28.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.417+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac19a430 con 0x7f41ac1020b0 2024-05-14T16:53:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.418+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac103590 con 0x7f41ac1020b0 2024-05-14T16:53:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.418+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f419c024020 con 0x7f41ac1020b0 2024-05-14T16:53:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.419+0000 7f41a8ff9700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f419803e510 0x7f41980409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.419+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f419c00f050 con 0x7f41ac1020b0 2024-05-14T16:53:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.419+0000 7f41a27fc700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f419803e510 0x7f41980409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.419+0000 7f41a27fc700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f419803e510 0x7f41980409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4194006fd0 tx=0x7f4194006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.423+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f419c02b020 con 0x7f41ac1020b0 2024-05-14T16:53:28.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:28.629+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7f41ac064ba0 con 0x7f419803e510 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: Deploying cephadm binary to smithi193 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:30.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:53:30.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:30.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:30 smithi099 ceph-mon[31851]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T16:53:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:30 smithi099 ceph-mon[31851]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:31 smithi099 ceph-mon[31851]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:32.697 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-05-14T16:53:32.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.697+0000 7f41a8ff9700 1 -- 172.21.15.99:0/819510074 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f41ac064ba0 con 0x7f419803e510 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f419803e510 msgr2=0x7f41980409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f419803e510 0x7f41980409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4194006fd0 tx=0x7f4194006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 msgr2=0x7f41ac1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac1997a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f419c006130 tx=0x7f419c0097d0 comp rx=0 tx=0).stop 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 shutdown_connections 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f419803e510 0x7f41980409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 --2- 172.21.15.99:0/819510074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ac1020b0 0x7f41ac1997a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:32.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 >> 172.21.15.99:0/819510074 conn(0x7f41ac0fd960 msgr2=0x7f41ac103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:32.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 shutdown_connections 2024-05-14T16:53:32.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:32.699+0000 7f41b1484700 1 -- 172.21.15.99:0/819510074 wait complete. 2024-05-14T16:53:32.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:32 smithi099 ceph-mon[31851]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:32.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:32.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:32.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:32.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:32.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:33.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T16:53:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:34.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:34 smithi099 ceph-mon[31851]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:34.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:34 smithi099 ceph-mon[31851]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:53:34.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:34 smithi099 ceph-mon[31851]: Added host smithi193 2024-05-14T16:53:34.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:34 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:34.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:34 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:34.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.981+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1653896318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 msgr2=0x7fdccc105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:34.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.981+0000 7fdcd2d68700 1 --2- 172.21.15.99:0/1653896318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc105310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009910 tx=0x7fdcc0009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:34.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.981+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1653896318 shutdown_connections 2024-05-14T16:53:34.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.981+0000 7fdcd2d68700 1 --2- 172.21.15.99:0/1653896318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc105310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.981+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1653896318 >> 172.21.15.99:0/1653896318 conn(0x7fdccc0fd0b0 msgr2=0x7fdccc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1653896318 shutdown_connections 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1653896318 wait complete. 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd2d68700 1 Processor -- start 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd2d68700 1 -- start start 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd2d68700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd2d68700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc10d620 con 0x7fdccc104f30 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd0b04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd0b04700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42772/0 (socket says 172.21.15.99:42772) 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.982+0000 7fdcd0b04700 1 -- 172.21.15.99:0/1992672824 learned_addr learned my addr 172.21.15.99:0/1992672824 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcd0b04700 1 -- 172.21.15.99:0/1992672824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc19ec70 con 0x7fdccc104f30 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcd0b04700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc19e730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00056d0 tx=0x7fdcc00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcc0026070 con 0x7fdccc104f30 2024-05-14T16:53:34.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc000bc00 con 0x7fdccc104f30 2024-05-14T16:53:34.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc19ef00 con 0x7fdccc104f30 2024-05-14T16:53:34.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcc0003920 con 0x7fdccc104f30 2024-05-14T16:53:34.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.983+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc1988f0 con 0x7fdccc104f30 2024-05-14T16:53:34.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.984+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fdcc0029070 con 0x7fdccc104f30 2024-05-14T16:53:34.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.984+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdccc1047b0 con 0x7fdccc104f30 2024-05-14T16:53:34.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.984+0000 7fdcc9ffb700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdcbc03e510 0x7fdcbc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:34.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.984+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdcc001e070 con 0x7fdccc104f30 2024-05-14T16:53:34.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.985+0000 7fdccbfff700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdcbc03e510 0x7fdcbc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:34.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.985+0000 7fdccbfff700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdcbc03e510 0x7fdcbc0409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006fd0 tx=0x7fdcb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:34.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:34.990+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcc0013070 con 0x7fdccc104f30 2024-05-14T16:53:35.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.197+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdccc064ba0 con 0x7fdcbc03e510 2024-05-14T16:53:35.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.198+0000 7fdcc9ffb700 1 -- 172.21.15.99:0/1992672824 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fdccc064ba0 con 0x7fdcbc03e510 2024-05-14T16:53:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:53:35.198 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-05-14T16:53:35.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdcbc03e510 msgr2=0x7fdcbc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:35.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdcbc03e510 0x7fdcbc0409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006fd0 tx=0x7fdcb8006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:35.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 msgr2=0x7fdccc19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc19e730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00056d0 tx=0x7fdcc00057b0 comp rx=0 tx=0).stop 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 shutdown_connections 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdcbc03e510 0x7fdcbc0409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 --2- 172.21.15.99:0/1992672824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc104f30 0x7fdccc19e730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.200+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 >> 172.21.15.99:0/1992672824 conn(0x7fdccc0fd0b0 msgr2=0x7fdccc10ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.201+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 shutdown_connections 2024-05-14T16:53:35.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:35.201+0000 7fdcd2d68700 1 -- 172.21.15.99:0/1992672824 wait complete. 2024-05-14T16:53:35.860 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-14T16:53:35.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-14T16:53:36.094 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:36.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:36 smithi099 ceph-mon[31851]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:36.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:36 smithi099 ceph-mon[31851]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:36.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:36 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.345+0000 7f3f446f8700 1 -- 172.21.15.99:0/3471663497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 msgr2=0x7f3f3c108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.345+0000 7f3f446f8700 1 --2- 172.21.15.99:0/3471663497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c108370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009910 tx=0x7f3f38009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 -- 172.21.15.99:0/3471663497 shutdown_connections 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 --2- 172.21.15.99:0/3471663497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c108370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 -- 172.21.15.99:0/3471663497 >> 172.21.15.99:0/3471663497 conn(0x7f3f3c0fd0f0 msgr2=0x7f3f3c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 -- 172.21.15.99:0/3471663497 shutdown_connections 2024-05-14T16:53:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 -- 172.21.15.99:0/3471663497 wait complete. 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 Processor -- start 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 -- start start 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.346+0000 7f3f446f8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c10d5a0 con 0x7f3f3c107f90 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f42494700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f42494700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42796/0 (socket says 172.21.15.99:42796) 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f42494700 1 -- 172.21.15.99:0/1390211232 learned_addr learned my addr 172.21.15.99:0/1390211232 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f42494700 1 -- 172.21.15.99:0/1390211232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c19ebe0 con 0x7f3f3c107f90 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f42494700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c19e6a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f380056d0 tx=0x7f3f380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38026070 con 0x7f3f3c107f90 2024-05-14T16:53:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f3800bc00 con 0x7f3f3c107f90 2024-05-14T16:53:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c19ee70 con 0x7f3f3c107f90 2024-05-14T16:53:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38003920 con 0x7f3f3c107f90 2024-05-14T16:53:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.347+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c1988c0 con 0x7f3f3c107f90 2024-05-14T16:53:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.348+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c1046d0 con 0x7f3f3c107f90 2024-05-14T16:53:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.348+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3f38029070 con 0x7f3f3c107f90 2024-05-14T16:53:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.348+0000 7f3f337fe700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3f2803e510 0x7f3f280409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.348+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f3801e070 con 0x7f3f3c107f90 2024-05-14T16:53:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.349+0000 7f3f41c93700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3f2803e510 0x7f3f280409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.349+0000 7f3f41c93700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3f2803e510 0x7f3f280409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006fd0 tx=0x7f3f2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:37.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.353+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f38013070 con 0x7f3f3c107f90 2024-05-14T16:53:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:37.562+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3f3c064ba0 con 0x7f3f3c107f90 2024-05-14T16:53:38.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.253+0000 7f3f337fe700 1 -- 172.21.15.99:0/1390211232 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3f3800bdb0 con 0x7f3f3c107f90 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3f2803e510 msgr2=0x7f3f280409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3f2803e510 0x7f3f280409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006fd0 tx=0x7f3f2c006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 msgr2=0x7f3f3c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c19e6a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f380056d0 tx=0x7f3f380057b0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 shutdown_connections 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f3f2803e510 0x7f3f280409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 --2- 172.21.15.99:0/1390211232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f3c107f90 0x7f3f3c19e6a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.256+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 >> 172.21.15.99:0/1390211232 conn(0x7f3f3c0fd0f0 msgr2=0x7f3f3c10ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:38.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.257+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 shutdown_connections 2024-05-14T16:53:38.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:53:38.257+0000 7f3f446f8700 1 -- 172.21.15.99:0/1390211232 wait complete. 2024-05-14T16:53:38.257 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2024-05-14T16:53:38.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:38 smithi099 ceph-mon[31851]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:38.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:38 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1390211232' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T16:53:38.867 INFO:tasks.cephadm:Adding mon.a on smithi099 2024-05-14T16:53:38.867 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-05-14T16:53:38.867 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-05-14T16:53:38.867 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c' 2024-05-14T16:53:39.100 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1390211232' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T16:53:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:53:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:39.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:53:39.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:39.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:40.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:40.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T16:53:40.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:40.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:40.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:53:40.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:40.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:40.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:40.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.644+0000 7ff7832a4700 1 -- 172.21.15.193:0/2301034449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 msgr2=0x7ff77c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.644+0000 7ff7832a4700 1 --2- 172.21.15.193:0/2301034449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c103590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b0a0 tx=0x7ff76c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.644+0000 7ff7832a4700 1 -- 172.21.15.193:0/2301034449 shutdown_connections 2024-05-14T16:53:40.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.644+0000 7ff7832a4700 1 --2- 172.21.15.193:0/2301034449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c103590 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.645 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.644+0000 7ff7832a4700 1 -- 172.21.15.193:0/2301034449 >> 172.21.15.193:0/2301034449 conn(0x7ff77c0fea80 msgr2=0x7ff77c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:40.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.645+0000 7ff7832a4700 1 -- 172.21.15.193:0/2301034449 shutdown_connections 2024-05-14T16:53:40.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.645+0000 7ff7832a4700 1 -- 172.21.15.193:0/2301034449 wait complete. 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.645+0000 7ff7832a4700 1 Processor -- start 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.645+0000 7ff7832a4700 1 -- start start 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.645+0000 7ff7832a4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.645+0000 7ff7832a4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10d430 con 0x7ff77c1031b0 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff781040700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff781040700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.193:47728/0 (socket says 172.21.15.193:47728) 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff781040700 1 -- 172.21.15.193:0/1422627454 learned_addr learned my addr 172.21.15.193:0/1422627454 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:53:40.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff781040700 1 -- 172.21.15.193:0/1422627454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c19ebb0 con 0x7ff77c1031b0 2024-05-14T16:53:40.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff781040700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c19e670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006130 tx=0x7ff76c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:40.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76c021020 con 0x7ff77c1031b0 2024-05-14T16:53:40.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c19ee40 con 0x7ff77c1031b0 2024-05-14T16:53:40.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff76c0079b0 con 0x7ff77c1031b0 2024-05-14T16:53:40.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76c017040 con 0x7ff77c1031b0 2024-05-14T16:53:40.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.646+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c198740 con 0x7ff77c1031b0 2024-05-14T16:53:40.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.647+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c104690 con 0x7ff77c1031b0 2024-05-14T16:53:40.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.647+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff76c024020 con 0x7ff77c1031b0 2024-05-14T16:53:40.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.647+0000 7ff77a7fc700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff76403e510 0x7ff7640409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:40.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.647+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff76c00f050 con 0x7ff77c1031b0 2024-05-14T16:53:40.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.648+0000 7ff78083f700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff76403e510 0x7ff7640409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:40.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.648+0000 7ff78083f700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff76403e510 0x7ff7640409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff770006fd0 tx=0x7ff770006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:40.653 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.653+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff76c02b020 con 0x7ff77c1031b0 2024-05-14T16:53:40.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.866+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) -- 0x7ff77c199d30 con 0x7ff76403e510 2024-05-14T16:53:40.874 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.873+0000 7ff77a7fc700 1 -- 172.21.15.193:0/1422627454 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff77c199d30 con 0x7ff76403e510 2024-05-14T16:53:40.874 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-05-14T16:53:40.876 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff76403e510 msgr2=0x7ff7640409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.876 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff76403e510 0x7ff7640409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff770006fd0 tx=0x7ff770006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:40.876 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 msgr2=0x7ff77c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.876 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c19e670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006130 tx=0x7ff76c0097d0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.877 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 shutdown_connections 2024-05-14T16:53:40.877 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff76403e510 0x7ff7640409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.877 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 --2- 172.21.15.193:0/1422627454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c1031b0 0x7ff77c19e670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.877 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 >> 172.21.15.193:0/1422627454 conn(0x7ff77c0fea80 msgr2=0x7ff77c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:40.877 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 shutdown_connections 2024-05-14T16:53:40.877 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:40.875+0000 7ff7832a4700 1 -- 172.21.15.193:0/1422627454 wait complete. 2024-05-14T16:53:41.475 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.b.service 2024-05-14T16:53:41.478 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.c.service 2024-05-14T16:53:41.480 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:53:41.480 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:53:42.237 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c;count:3 2024-05-14T16:53:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:42.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:42.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:42.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:53:42.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:42.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:41 smithi099 ceph-mon[31851]: Deploying daemon mon.c on smithi193 2024-05-14T16:53:44.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 -- 172.21.15.193:0/4253681185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 msgr2=0x7f4854102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:44.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 --2- 172.21.15.193:0/4253681185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f4854102490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4844009910 tx=0x7f4844009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 -- 172.21.15.193:0/4253681185 shutdown_connections 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 --2- 172.21.15.193:0/4253681185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f4854102490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 -- 172.21.15.193:0/4253681185 >> 172.21.15.193:0/4253681185 conn(0x7f48540fd960 msgr2=0x7f48540ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 -- 172.21.15.193:0/4253681185 shutdown_connections 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.082+0000 7f4858ff7700 1 -- 172.21.15.193:0/4253681185 wait complete. 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.083+0000 7f4858ff7700 1 Processor -- start 2024-05-14T16:53:44.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.083+0000 7f4858ff7700 1 -- start start 2024-05-14T16:53:44.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.083+0000 7f4858ff7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f485419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:44.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.083+0000 7f4858ff7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f485410c330 con 0x7f48541020b0 2024-05-14T16:53:44.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f485259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f485419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:44.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f485259c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f485419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.193:47742/0 (socket says 172.21.15.193:47742) 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f485259c700 1 -- 172.21.15.193:0/2758344941 learned_addr learned my addr 172.21.15.193:0/2758344941 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f485259c700 1 -- 172.21.15.193:0/2758344941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485419dba0 con 0x7f48541020b0 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f485259c700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f485419d660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f48440056d0 tx=0x7f48440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4844026070 con 0x7f48541020b0 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485419de90 con 0x7f48541020b0 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f484400bba0 con 0x7f48541020b0 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4844003920 con 0x7f48541020b0 2024-05-14T16:53:44.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.084+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4854197900 con 0x7f48541020b0 2024-05-14T16:53:44.087 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.085+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4844029070 con 0x7f48541020b0 2024-05-14T16:53:44.087 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.085+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4854103590 con 0x7f48541020b0 2024-05-14T16:53:44.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.086+0000 7f48437fe700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f483c03e510 0x7f483c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:44.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.086+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f484401e070 con 0x7f48541020b0 2024-05-14T16:53:44.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.086+0000 7f4851d9b700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f483c03e510 0x7f483c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:44.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.086+0000 7f4851d9b700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f483c03e510 0x7f483c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4848006fd0 tx=0x7f4848006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:44.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.091+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4844013070 con 0x7f48541020b0 2024-05-14T16:53:44.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:43 smithi099 ceph-mon[31851]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:44.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.385+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f48540008d0 con 0x7f48541020b0 2024-05-14T16:53:44.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.385+0000 7f48437fe700 1 -- 172.21.15.193:0/2758344941 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4844030020 con 0x7f48541020b0 2024-05-14T16:53:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:53:44.387 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","modified":"2024-05-14T16:52:05.874052Z","created":"2024-05-14T16:52:05.874052Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-14T16:53:44.388 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f483c03e510 msgr2=0x7f483c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f483c03e510 0x7f483c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4848006fd0 tx=0x7f4848006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 msgr2=0x7f485419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f485419d660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f48440056d0 tx=0x7f48440057b0 comp rx=0 tx=0).stop 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 shutdown_connections 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f483c03e510 0x7f483c0409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 --2- 172.21.15.193:0/2758344941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48541020b0 0x7f485419d660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 >> 172.21.15.193:0/2758344941 conn(0x7f48540fd960 msgr2=0x7f4854103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 shutdown_connections 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:44.388+0000 7f4858ff7700 1 -- 172.21.15.193:0/2758344941 wait complete. 2024-05-14T16:53:44.389 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-14T16:53:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:44 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/2758344941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:53:45.948 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:53:45.949 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:53:46.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:45 smithi099 ceph-mon[31851]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:46.328 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:53:46.452 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:46 smithi193 systemd[1]: Starting Ceph mon.c for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 -- 172.21.15.193:0/2260001152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 msgr2=0x7f4534102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 --2- 172.21.15.193:0/2260001152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534102570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4524009910 tx=0x7f4524009c20 comp rx=0 tx=0).stop 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 -- 172.21.15.193:0/2260001152 shutdown_connections 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 --2- 172.21.15.193:0/2260001152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534102570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 -- 172.21.15.193:0/2260001152 >> 172.21.15.193:0/2260001152 conn(0x7f45340fda20 msgr2=0x7f45340ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 -- 172.21.15.193:0/2260001152 shutdown_connections 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.682+0000 7f453be76700 1 -- 172.21.15.193:0/2260001152 wait complete. 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.683+0000 7f453be76700 1 Processor -- start 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.683+0000 7f453be76700 1 -- start start 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.683+0000 7f453be76700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:47.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.683+0000 7f453be76700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f453410c410 con 0x7f4534102190 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.683+0000 7f4539c12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f4539c12700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.193:47756/0 (socket says 172.21.15.193:47756) 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f4539c12700 1 -- 172.21.15.193:0/2901873815 learned_addr learned my addr 172.21.15.193:0/2901873815 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f4539c12700 1 -- 172.21.15.193:0/2901873815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4534199eb0 con 0x7f4534102190 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f4539c12700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534199970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45240056d0 tx=0x7f45240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4524026070 con 0x7f4534102190 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453419a1a0 con 0x7f4534102190 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452400bba0 con 0x7f4534102190 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4524003920 con 0x7f4534102190 2024-05-14T16:53:47.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.684+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453419a660 con 0x7f4534102190 2024-05-14T16:53:47.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.685+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4534103670 con 0x7f4534102190 2024-05-14T16:53:47.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.685+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4524029070 con 0x7f4534102190 2024-05-14T16:53:47.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.685+0000 7f452affd700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f452003e570 0x7f4520040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:47.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.686+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f452401e070 con 0x7f4534102190 2024-05-14T16:53:47.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.686+0000 7f4539411700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f452003e570 0x7f4520040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:47.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.686+0000 7f4539411700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f452003e570 0x7f4520040a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4530006fd0 tx=0x7f4530006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:47.691 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.691+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4524013070 con 0x7f4534102190 2024-05-14T16:53:47.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 podman[35914]: 2024-05-14 16:53:47.465272828 +0000 UTC m=+0.198403829 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:53:47.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 podman[35914]: 2024-05-14 16:53:47.734793569 +0000 UTC m=+0.467924603 container create 2e8dd14d2223cb36910c68b333735cb737dfb38b021d3d88e858045792737397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-14T16:53:47.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 podman[35914]: 2024-05-14 16:53:47.946051803 +0000 UTC m=+0.679182824 container init 2e8dd14d2223cb36910c68b333735cb737dfb38b021d3d88e858045792737397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-14T16:53:47.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 podman[35914]: 2024-05-14 16:53:47.949204667 +0000 UTC m=+0.682335676 container start 2e8dd14d2223cb36910c68b333735cb737dfb38b021d3d88e858045792737397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T16:53:47.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T16:53:47.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-14T16:53:47.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: pidfile_write: ignore empty --pid-file 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: load: jerasure load: lrc 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: RocksDB version: 7.9.2 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Git sha 0 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: DB SUMMARY 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: DB Session ID: EDKLWCRZKKW88CPP9AJH 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: CURRENT file: CURRENT 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.error_if_exists: 0 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.create_if_missing: 0 2024-05-14T16:53:47.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.paranoid_checks: 1 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.env: 0x55d8ef52acc0 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.info_log: 0x55d8f08c3da0 2024-05-14T16:53:47.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T16:53:47.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.statistics: (nil) 2024-05-14T16:53:47.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.use_fsync: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_log_file_size: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.allow_fallocate: 1 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.use_direct_reads: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.db_log_dir: 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.wal_dir: 2024-05-14T16:53:47.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.write_buffer_manager: 0x55d8f08950e0 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T16:53:47.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.unordered_write: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.row_cache: None 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.wal_filter: None 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.two_write_queues: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T16:53:47.990 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.wal_compression: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.atomic_flush: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.log_readahead_size: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_background_jobs: 2 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_background_compactions: -1 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_subcompactions: 1 2024-05-14T16:53:47.991 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_open_files: -1 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_background_flushes: -1 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Compression algorithms supported: 2024-05-14T16:53:47.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kZSTD supported: 0 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kXpressCompression supported: 0 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kZlibCompression supported: 1 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kSnappyCompression supported: 1 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kLZ4Compression supported: 1 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: kBZip2Compression supported: 0 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.merge_operator: 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_filter: None 2024-05-14T16:53:47.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8f08c3f60) 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-05-14T16:53:47.994 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x55d8f08c71f0 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-05-14T16:53:47.995 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T16:53:47.996 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression: NoCompression 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.num_levels: 7 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:53:47.997 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T16:53:47.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T16:53:47.999 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T16:53:48.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.inplace_update_support: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.bloom_locality: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.max_successive_merges: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.ttl: 2592000 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T16:53:48.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.enable_blob_files: false 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.min_blob_size: 0 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eb618b0b-71e1-49c3-ad8a-91e59d4c9c84 2024-05-14T16:53:48.002 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715705627979622, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715705627980200, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715705627, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eb618b0b-71e1-49c3-ad8a-91e59d4c9c84", "db_session_id": "EDKLWCRZKKW88CPP9AJH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715705627980307, "job": 1, "event": "recovery_finished"} 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d8f09e4000 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: DB pointer 0x55d8f09d0000 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:53:48.003 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T16:53:48.004 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x55d8f08c71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-14T16:53:48.005 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T16:53:48.006 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T16:53:48.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-14T16:53:48.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-05-14T16:53:48.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:48.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:47 smithi193 ceph-mon[35962]: mon.c@-1(???) e0 preinit fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:48.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.993+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4534064ba0 con 0x7f4534102190 2024-05-14T16:53:48.007 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.994+0000 7f452affd700 1 -- 172.21.15.193:0/2901873815 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4524030090 con 0x7f4534102190 2024-05-14T16:53:48.007 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:53:48.007 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","modified":"2024-05-14T16:52:05.874052Z","created":"2024-05-14T16:52:05.874052Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f452003e570 msgr2=0x7f4520040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f452003e570 0x7f4520040a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4530006fd0 tx=0x7f4530006e40 comp rx=0 tx=0).stop 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 msgr2=0x7f4534199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534199970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45240056d0 tx=0x7f45240057b0 comp rx=0 tx=0).stop 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 shutdown_connections 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f452003e570 0x7f4520040a30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 --2- 172.21.15.193:0/2901873815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4534102190 0x7f4534199970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 >> 172.21.15.193:0/2901873815 conn(0x7f45340fda20 msgr2=0x7f4534103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 shutdown_connections 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:47.996+0000 7f453be76700 1 -- 172.21.15.193:0/2901873815 wait complete. 2024-05-14T16:53:48.008 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-14T16:53:48.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:47 smithi099 ceph-mon[31851]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).mds e1 new map 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-14T16:53:48.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Deploying cephadm binary to smithi193 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:53:48.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Added host smithi193 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1390211232' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1390211232' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T16:53:48.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c;count:3 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: Deploying daemon mon.c on smithi193 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/2758344941' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:48.299 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 ceph-mon[35962]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-14T16:53:48.299 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 bash[35914]: 2e8dd14d2223cb36910c68b333735cb737dfb38b021d3d88e858045792737397 2024-05-14T16:53:48.299 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:48 smithi193 systemd[1]: Started Ceph mon.c for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:53:49.727 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:53:49.727 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:53:49.971 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: mon.a calling monitor election 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: mon.c calling monitor election 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: monmap epoch 2 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: last_changed 2024-05-14T16:53:48.006136+0000 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: created 2024-05-14T16:52:05.874052+0000 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: min_mon_release 19 (squid) 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: election_strategy: 1 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: 0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: fsmap 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: mgrmap e13: x(active, since 50s) 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: overall HEALTH_OK 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:53.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:53.517 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:53:53.517 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: mon.a calling monitor election 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: mon.c calling monitor election 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: monmap epoch 2 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: last_changed 2024-05-14T16:53:48.006136+0000 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: created 2024-05-14T16:52:05.874052+0000 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: min_mon_release 19 (squid) 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: election_strategy: 1 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: 0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: fsmap 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:53:53.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: mgrmap e13: x(active, since 50s) 2024-05-14T16:53:53.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: overall HEALTH_OK 2024-05-14T16:53:53.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:53.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:53.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:53:53.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:53:53.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:54.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 -- 172.21.15.193:0/2174401770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 msgr2=0x7f4770005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 --2- 172.21.15.193:0/2174401770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f4770005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4784101e80 tx=0x7f477400feb0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 -- 172.21.15.193:0/2174401770 shutdown_connections 2024-05-14T16:53:54.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 --2- 172.21.15.193:0/2174401770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f4770005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.395 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 -- 172.21.15.193:0/2174401770 >> 172.21.15.193:0/2174401770 conn(0x7f47840fda80 msgr2=0x7f47840ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.395 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 -- 172.21.15.193:0/2174401770 shutdown_connections 2024-05-14T16:53:54.395 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.394+0000 7f478bebf700 1 -- 172.21.15.193:0/2174401770 wait complete. 2024-05-14T16:53:54.395 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:54 smithi193 ceph-mon[35962]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:54.395 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:54 smithi193 ceph-mon[35962]: Deploying daemon mon.b on smithi150 2024-05-14T16:53:54.395 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:54 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.395+0000 7f478bebf700 1 Processor -- start 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.395+0000 7f478bebf700 1 -- start start 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.395+0000 7f478bebf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f478419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.395+0000 7f478bebf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.395+0000 7f478bebf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4784109ea0 con 0x7f47700035c0 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.395+0000 7f478bebf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4784109d20 con 0x7f478419db70 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:46372/0 (socket says 172.21.15.193:46372) 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 -- 172.21.15.193:0/1501861160 learned_addr learned my addr 172.21.15.193:0/1501861160 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:53:54.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f4789c5b700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f478419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 -- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 msgr2=0x7f47841977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 -- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 msgr2=0x7f47841977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f478945a700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f4789c5b700 1 -- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 msgr2=0x7f47841977b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f4789c5b700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f4789c5b700 1 -- 172.21.15.193:0/1501861160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478419e1b0 con 0x7f47700035c0 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f4789c5b700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f478419d630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4774010010 tx=0x7f4774005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.396+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4774004aa0 con 0x7f47700035c0 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.397+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4784197d50 con 0x7f47700035c0 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.397+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841982b0 con 0x7f47700035c0 2024-05-14T16:53:54.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.397+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4774004030 con 0x7f47700035c0 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.397+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4774011b30 con 0x7f47700035c0 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.398+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4774023070 con 0x7f47700035c0 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.398+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784103690 con 0x7f47700035c0 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.398+0000 7f477affd700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f476803e2b0 0x7f4768040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.398+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4774020050 con 0x7f47700035c0 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.398+0000 7f478945a700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f476803e2b0 0x7f4768040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.400 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.399+0000 7f478945a700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f476803e2b0 0x7f4768040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4780006b20 tx=0x7f4780006ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.403 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.404+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4774028020 con 0x7f47700035c0 2024-05-14T16:53:54.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:54 smithi099 ceph-mon[31851]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:54.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:54 smithi099 ceph-mon[31851]: Deploying daemon mon.b on smithi150 2024-05-14T16:53:54.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:54 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:53:54.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.693+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f47841985d0 con 0x7f47700035c0 2024-05-14T16:53:54.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.694+0000 7f477affd700 1 -- 172.21.15.193:0/1501861160 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f47740042e0 con 0x7f47700035c0 2024-05-14T16:53:54.695 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:53:54.695 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","modified":"2024-05-14T16:53:48.006136Z","created":"2024-05-14T16:52:05.874052Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T16:53:54.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f476803e2b0 msgr2=0x7f4768040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f476803e2b0 0x7f4768040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4780006b20 tx=0x7f4780006ab0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 msgr2=0x7f478419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f478419d630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4774010010 tx=0x7f4774005020 comp rx=0 tx=0).stop 2024-05-14T16:53:54.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 shutdown_connections 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f476803e2b0 0x7f4768040770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47700035c0 0x7f478419d630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 --2- 172.21.15.193:0/1501861160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f478419db70 0x7f47841977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 >> 172.21.15.193:0/1501861160 conn(0x7f47840fda80 msgr2=0x7f47840fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 shutdown_connections 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:53:54.696+0000 7f478bebf700 1 -- 172.21.15.193:0/1501861160 wait complete. 2024-05-14T16:53:54.698 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-05-14T16:53:55.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:55 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/1501861160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:53:55.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:55 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/1501861160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:53:56.342 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:53:56.342 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:53:56.369 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:53:56 smithi193 ceph-mon[35962]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:53:56 smithi099 ceph-mon[31851]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:53:56.584 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:53:57.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:53:57 smithi150 systemd[1]: Started Ceph mon.b for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:54:02.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:54:02.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: mon.a calling monitor election 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: mon.c calling monitor election 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: mon.b calling monitor election 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: monmap epoch 3 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:54:02.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: last_changed 2024-05-14T16:53:57.053439+0000 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: created 2024-05-14T16:52:05.874052+0000 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: min_mon_release 19 (squid) 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: election_strategy: 1 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: 0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: fsmap 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: mgrmap e13: x(active, since 59s) 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: overall HEALTH_OK 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.455 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 -- 172.21.15.193:0/194259254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86540035c0 msgr2=0x7f8654005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 --2- 172.21.15.193:0/194259254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86540035c0 0x7f8654005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8668101e60 tx=0x7f8660009e80 comp rx=0 tx=0).stop 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 -- 172.21.15.193:0/194259254 shutdown_connections 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 --2- 172.21.15.193:0/194259254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86540035c0 0x7f8654005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 -- 172.21.15.193:0/194259254 >> 172.21.15.193:0/194259254 conn(0x7f86680fda20 msgr2=0x7f86680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 -- 172.21.15.193:0/194259254 shutdown_connections 2024-05-14T16:54:02.457 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.457+0000 7f866f1c0700 1 -- 172.21.15.193:0/194259254 wait complete. 2024-05-14T16:54:02.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: mon.a calling monitor election 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: mon.c calling monitor election 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: mon.b calling monitor election 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: monmap epoch 3 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: last_changed 2024-05-14T16:53:57.053439+0000 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: created 2024-05-14T16:52:05.874052+0000 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: min_mon_release 19 (squid) 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: election_strategy: 1 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: 0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: fsmap 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: mgrmap e13: x(active, since 59s) 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: overall HEALTH_OK 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.459 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.460 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.458+0000 7f866f1c0700 1 Processor -- start 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.458+0000 7f866f1c0700 1 -- start start 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866f1c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 0x7f8668103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866f1c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866f1c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86681a4d90 0x7f86681a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866f1c0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8668109e20 con 0x7f86681a4d90 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866f1c0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8668109ca0 con 0x7f86540035c0 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866f1c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8668109fa0 con 0x7f8668104170 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866cf5c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 0x7f8668103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:48102/0 (socket says 172.21.15.193:48102) 2024-05-14T16:54:02.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 -- 172.21.15.193:0/1496422783 learned_addr learned my addr 172.21.15.193:0/1496422783 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86681a4d90 0x7f86681a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 msgr2=0x7f86681045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 msgr2=0x7f86681045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866cf5c700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 msgr2=0x7f8668103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866cf5c700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 msgr2=0x7f8668103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866cf5c700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 0x7f8668103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866cf5c700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 0x7f8668103c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f8667fff700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 msgr2=0x7f86681045f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 msgr2=0x7f8668103c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 0x7f8668103c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 -- 172.21.15.193:0/1496422783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86681a7850 con 0x7f86681a4d90 2024-05-14T16:54:02.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.459+0000 7f866d75d700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86681a4d90 0x7f86681a7180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8650006fd0 tx=0x7f865000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.460+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650015020 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.460+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8650011ce0 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.460+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650010d30 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.460+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86681a7b40 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.460+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86681a8080 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.461+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8650018020 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.461+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8668064ba0 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.461+0000 7f8665ffb700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f864803e9a0 0x7f8648040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.461+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8650052120 con 0x7f86681a4d90 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.461+0000 7f866cf5c700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f864803e9a0 0x7f8648040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.462+0000 7f866cf5c700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f864803e9a0 0x7f8648040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8660009a30 tx=0x7f866000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:02.466 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.467+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f865001f020 con 0x7f86681a4d90 2024-05-14T16:54:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:54:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: mon.a calling monitor election 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: mon.c calling monitor election 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: mon.b calling monitor election 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: monmap epoch 3 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: last_changed 2024-05-14T16:53:57.053439+0000 2024-05-14T16:54:02.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: created 2024-05-14T16:52:05.874052+0000 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: min_mon_release 19 (squid) 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: election_strategy: 1 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: 0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: fsmap 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: mgrmap e13: x(active, since 59s) 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: overall HEALTH_OK 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:02.759 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.759+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f866802e720 con 0x7f86681a4d90 2024-05-14T16:54:02.759 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.760+0000 7f8665ffb700 1 -- 172.21.15.193:0/1496422783 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f86500187b0 con 0x7f86681a4d90 2024-05-14T16:54:02.760 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T16:54:02.760 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","modified":"2024-05-14T16:53:57.053439Z","created":"2024-05-14T16:52:05.874052Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f864803e9a0 msgr2=0x7f8648040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f864803e9a0 0x7f8648040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8660009a30 tx=0x7f866000b040 comp rx=0 tx=0).stop 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86681a4d90 msgr2=0x7f86681a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86681a4d90 0x7f86681a7180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8650006fd0 tx=0x7f865000b7e0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 shutdown_connections 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f864803e9a0 0x7f8648040e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86681a4d90 0x7f86681a7180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86540035c0 0x7f8668103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 --2- 172.21.15.193:0/1496422783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8668104170 0x7f86681045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 >> 172.21.15.193:0/1496422783 conn(0x7f86680fda20 msgr2=0x7f86680fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:02.764 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 shutdown_connections 2024-05-14T16:54:02.764 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:02.763+0000 7f866f1c0700 1 -- 172.21.15.193:0/1496422783 wait complete. 2024-05-14T16:54:02.764 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-05-14T16:54:03.332 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-14T16:54:03.332 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-14T16:54:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:03 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/1496422783' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:54:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:03 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/1496422783' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:54:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:03 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/1496422783' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:54:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:04.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:04.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 -- 172.21.15.99:0/3115025904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 msgr2=0x7f9de4103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 --2- 172.21.15.99:0/3115025904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de4103590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009910 tx=0x7f9dd4009c20 comp rx=0 tx=0).stop 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 -- 172.21.15.99:0/3115025904 shutdown_connections 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 --2- 172.21.15.99:0/3115025904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de4103590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 -- 172.21.15.99:0/3115025904 >> 172.21.15.99:0/3115025904 conn(0x7f9de40fea60 msgr2=0x7f9de4100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 -- 172.21.15.99:0/3115025904 shutdown_connections 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.109+0000 7f9deb570700 1 -- 172.21.15.99:0/3115025904 wait complete. 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 Processor -- start 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 -- start start 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de419edd0 0x7f9de4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9de4198f90 0x7f9de4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9de410d430 con 0x7f9de41031b0 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9de410d2b0 con 0x7f9de419edd0 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9deb570700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9de410d5b0 con 0x7f9de4198f90 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9de930c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9de930c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48372/0 (socket says 172.21.15.99:48372) 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9de930c700 1 -- 172.21.15.99:0/1178018029 learned_addr learned my addr 172.21.15.99:0/1178018029 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9de8b0b700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de419edd0 0x7f9de4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.110+0000 7f9de9b0d700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9de4198f90 0x7f9de4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de930c700 1 -- 172.21.15.99:0/1178018029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9de4198f90 msgr2=0x7f9de4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de930c700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9de4198f90 0x7f9de4199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de930c700 1 -- 172.21.15.99:0/1178018029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de419edd0 msgr2=0x7f9de4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de930c700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de419edd0 0x7f9de4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de930c700 1 -- 172.21.15.99:0/1178018029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de41a5760 con 0x7f9de41031b0 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de9b0d700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9de4198f90 0x7f9de4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:05.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de8b0b700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de419edd0 0x7f9de4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:05.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9de930c700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de419e890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4005b40 tx=0x7f9dd4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:05.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4026070 con 0x7f9de41031b0 2024-05-14T16:54:05.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de41a5990 con 0x7f9de41031b0 2024-05-14T16:54:05.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd400b6f0 con 0x7f9de41031b0 2024-05-14T16:54:05.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40038e0 con 0x7f9de41031b0 2024-05-14T16:54:05.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.111+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de41a5f20 con 0x7f9de41031b0 2024-05-14T16:54:05.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.112+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9dd4029070 con 0x7f9de41031b0 2024-05-14T16:54:05.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.112+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de4064ba0 con 0x7f9de41031b0 2024-05-14T16:54:05.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.112+0000 7f9dda7fc700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9dd003e9a0 0x7f9dd0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.112+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9dd4053610 con 0x7f9de41031b0 2024-05-14T16:54:05.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.112+0000 7f9de8b0b700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9dd003e9a0 0x7f9dd0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.113+0000 7f9de8b0b700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9dd003e9a0 0x7f9dd0040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009910 tx=0x7f9de0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:05.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.118+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9dd4013070 con 0x7f9de41031b0 2024-05-14T16:54:05.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:05.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:05.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:05.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.332+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9de41a61b0 con 0x7f9de41031b0 2024-05-14T16:54:05.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.333+0000 7f9dda7fc700 1 -- 172.21.15.99:0/1178018029 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f9dd4030090 con 0x7f9de41031b0 2024-05-14T16:54:05.332 INFO:teuthology.orchestra.run.smithi099.stdout:# minimal ceph.conf for 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:54:05.332 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-05-14T16:54:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T16:54:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: mon_host = [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9dd003e9a0 msgr2=0x7f9dd0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9dd003e9a0 0x7f9dd0040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009910 tx=0x7f9de0008040 comp rx=0 tx=0).stop 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 msgr2=0x7f9de419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de419e890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4005b40 tx=0x7f9dd4005030 comp rx=0 tx=0).stop 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 shutdown_connections 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9dd003e9a0 0x7f9dd0040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9de4198f90 0x7f9de4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de41031b0 0x7f9de419e890 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 --2- 172.21.15.99:0/1178018029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de419edd0 0x7f9de4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 >> 172.21.15.99:0/1178018029 conn(0x7f9de40fea60 msgr2=0x7f9de4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 shutdown_connections 2024-05-14T16:54:05.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:05.335+0000 7f9deb570700 1 -- 172.21.15.99:0/1178018029 wait complete. 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:05.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T16:54:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T16:54:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T16:54:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:06.009 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-14T16:54:06.010 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:54:06.010 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:54:06.036 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:54:06.036 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:06.100 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:54:06.101 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:54:06.129 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:54:06.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:06.195 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:54:06.195 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:54:06.227 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:54:06.227 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:06.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:06 smithi099 ceph-mon[31851]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:06.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:06 smithi099 ceph-mon[31851]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:54:06.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:06 smithi099 ceph-mon[31851]: Reconfiguring daemon mon.a on smithi099 2024-05-14T16:54:06.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:06 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1178018029' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:06.294 INFO:tasks.cephadm:Adding mgr.x on smithi099 2024-05-14T16:54:06.294 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-05-14T16:54:06.295 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-05-14T16:54:06.295 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi099=x;smithi150=y;smithi193=z' 2024-05-14T16:54:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:06 smithi150 ceph-mon[36204]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:06 smithi150 ceph-mon[36204]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:54:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:06 smithi150 ceph-mon[36204]: Reconfiguring daemon mon.a on smithi099 2024-05-14T16:54:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:06 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1178018029' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:06 smithi193 ceph-mon[35962]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:06 smithi193 ceph-mon[35962]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:54:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:06 smithi193 ceph-mon[35962]: Reconfiguring daemon mon.a on smithi099 2024-05-14T16:54:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:06 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1178018029' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:06.569 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:07.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:07 smithi150 ceph-mon[36204]: Reconfiguring daemon mon.b on smithi150 2024-05-14T16:54:07.956 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 -- 172.21.15.193:0/1394644580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8974101fe0 msgr2=0x7f8974102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:07.956 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 --2- 172.21.15.193:0/1394644580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8974101fe0 0x7f8974102400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8970009910 tx=0x7f8970009c20 comp rx=0 tx=0).stop 2024-05-14T16:54:07.956 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 -- 172.21.15.193:0/1394644580 shutdown_connections 2024-05-14T16:54:07.956 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 --2- 172.21.15.193:0/1394644580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8974101fe0 0x7f8974102400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 -- 172.21.15.193:0/1394644580 >> 172.21.15.193:0/1394644580 conn(0x7f89740fd580 msgr2=0x7f89740ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 -- 172.21.15.193:0/1394644580 shutdown_connections 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.953+0000 7f897bb7a700 1 -- 172.21.15.193:0/1394644580 wait complete. 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 Processor -- start 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 -- start start 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974101fe0 0x7f897406c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897419b560 0x7f897419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 0x7f89741a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8974107f20 con 0x7f897419b560 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8974107da0 con 0x7f897419df30 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897bb7a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89741080a0 con 0x7f8974101fe0 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897a117700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 0x7f89741a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897a117700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 0x7f89741a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.193:48940/0 (socket says 172.21.15.193:48940) 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f897a117700 1 -- 172.21.15.193:0/3435359031 learned_addr learned my addr 172.21.15.193:0/3435359031 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f8979916700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974101fe0 0x7f897406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:07.957 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.954+0000 7f8979115700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897419b560 0x7f897419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897a117700 1 -- 172.21.15.193:0/3435359031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974101fe0 msgr2=0x7f897406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897a117700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974101fe0 0x7f897406c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897a117700 1 -- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897419b560 msgr2=0x7f897419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897a117700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897419b560 0x7f897419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897a117700 1 -- 172.21.15.193:0/3435359031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89741a0c90 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f8979916700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974101fe0 0x7f897406c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f8979115700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897419b560 0x7f897419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897a117700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 0x7f89741a03c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f896c00ba50 tx=0x7f896c00bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c019070 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896c00cde0 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89741973a0 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c007350 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.955+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8974197960 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.956+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f896c01c070 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.956+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8974064ba0 con 0x7f897419df30 2024-05-14T16:54:07.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.956+0000 7f896affd700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f896003e9a0 0x7f8960040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:07.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.956+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f896c051ea0 con 0x7f897419df30 2024-05-14T16:54:07.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.956+0000 7f8979916700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f896003e9a0 0x7f8960040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:07.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.957+0000 7f8979916700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f896003e9a0 0x7f8960040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8970005850 tx=0x7f8970005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:07.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:07.962+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f896c013a90 con 0x7f897419df30 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:07.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:07.962 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:07 smithi193 ceph-mon[35962]: Reconfiguring daemon mon.b on smithi150 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:07 smithi099 ceph-mon[31851]: Reconfiguring daemon mon.b on smithi150 2024-05-14T16:54:08.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.180+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi150=y;smithi193=z", "target": ["mon-mgr", ""]}) -- 0x7f8974063a40 con 0x7f896003e9a0 2024-05-14T16:54:08.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.187+0000 7f896affd700 1 -- 172.21.15.193:0/3435359031 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8974063a40 con 0x7f896003e9a0 2024-05-14T16:54:08.186 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-05-14T16:54:08.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f896003e9a0 msgr2=0x7f8960040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f896003e9a0 0x7f8960040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8970005850 tx=0x7f8970005260 comp rx=0 tx=0).stop 2024-05-14T16:54:08.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 msgr2=0x7f89741a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 0x7f89741a03c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f896c00ba50 tx=0x7f896c00bd60 comp rx=0 tx=0).stop 2024-05-14T16:54:08.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 shutdown_connections 2024-05-14T16:54:08.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f896003e9a0 0x7f8960040e60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897419df30 0x7f89741a03c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974101fe0 0x7f897406c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 --2- 172.21.15.193:0/3435359031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897419b560 0x7f897419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 >> 172.21.15.193:0/3435359031 conn(0x7f89740fd580 msgr2=0x7f89740fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 shutdown_connections 2024-05-14T16:54:08.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:54:08.189+0000 7f897bb7a700 1 -- 172.21.15.193:0/3435359031 wait complete. 2024-05-14T16:54:08.723 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.y.service 2024-05-14T16:54:08.726 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.z.service 2024-05-14T16:54:08.728 INFO:tasks.cephadm:Deploying OSDs... 2024-05-14T16:54:08.728 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T16:54:08.728 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:54:08.748 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:54:08.749 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-14 16:53:12.370007936 +0000 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-14 16:46:56.805896826 +0000 2024-05-14T16:54:08.808 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-14 16:46:56.805896826 +0000 2024-05-14T16:54:08.809 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-14 16:46:56.805896826 +0000 2024-05-14T16:54:08.809 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:54:08.876 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-14T16:54:08.876 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-14T16:54:08.876 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000235025 s, 2.2 MB/s 2024-05-14T16:54:08.877 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:54:08.935 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-05-14T16:54:08.992 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:54:08.992 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:08.992 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-14T16:54:08.992 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:08.993 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:08.993 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-14 16:53:12.407007105 +0000 2024-05-14T16:54:08.993 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-14 16:46:57.148888936 +0000 2024-05-14T16:54:08.993 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-14 16:46:57.148888936 +0000 2024-05-14T16:54:08.993 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-14 16:46:57.148888936 +0000 2024-05-14T16:54:08.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:54:09.057 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-14T16:54:09.057 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-14T16:54:09.058 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000273238 s, 1.9 MB/s 2024-05-14T16:54:09.059 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:54:09.116 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-14 16:53:12.441006342 +0000 2024-05-14T16:54:09.172 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-14 16:46:57.478881344 +0000 2024-05-14T16:54:09.173 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-14 16:46:57.478881344 +0000 2024-05-14T16:54:09.173 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-14 16:46:57.478881344 +0000 2024-05-14T16:54:09.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:54:09.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: Reconfiguring daemon mon.c on smithi193 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi150=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: Saving service mgr spec with placement smithi099=x;smithi150=y;smithi193=z;count:3 2024-05-14T16:54:09.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.237 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-14T16:54:09.237 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-14T16:54:09.237 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000194261 s, 2.6 MB/s 2024-05-14T16:54:09.238 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:54:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: Reconfiguring daemon mon.c on smithi193 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi150=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: Saving service mgr spec with placement smithi099=x;smithi150=y;smithi193=z;count:3 2024-05-14T16:54:09.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.295 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-14 16:53:12.473005624 +0000 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-14 16:46:57.804873844 +0000 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-14 16:46:57.804873844 +0000 2024-05-14T16:54:09.351 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-14 16:46:57.804873844 +0000 2024-05-14T16:54:09.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:54:09.415 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-14T16:54:09.415 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-14T16:54:09.415 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000190696 s, 2.7 MB/s 2024-05-14T16:54:09.417 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: Reconfiguring daemon mon.c on smithi193 2024-05-14T16:54:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi150=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:09.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: Saving service mgr spec with placement smithi099=x;smithi150=y;smithi193=z;count:3 2024-05-14T16:54:09.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:09.474 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-14T16:54:09.474 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:54:09.493 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:54:09.493 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-14 16:53:56.537490397 +0000 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-14 16:48:20.065514819 +0000 2024-05-14T16:54:09.551 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-14 16:48:20.065514819 +0000 2024-05-14T16:54:09.552 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-14 16:48:20.065514819 +0000 2024-05-14T16:54:09.552 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:54:09.617 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-14T16:54:09.617 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-14T16:54:09.617 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000270664 s, 1.9 MB/s 2024-05-14T16:54:09.618 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:54:09.677 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-05-14T16:54:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:54:09.733 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-14 16:53:56.537490397 +0000 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-14 16:48:20.414508329 +0000 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-14 16:48:20.414508329 +0000 2024-05-14T16:54:09.734 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-14 16:48:20.414508329 +0000 2024-05-14T16:54:09.734 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:54:09.800 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-14T16:54:09.801 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-14T16:54:09.801 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000230657 s, 2.2 MB/s 2024-05-14T16:54:09.802 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:54:09.859 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-05-14T16:54:09.915 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:54:09.915 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:09.915 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-14T16:54:09.915 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:09.915 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:09.916 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-14 16:53:56.537490397 +0000 2024-05-14T16:54:09.916 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-14 16:48:20.761501877 +0000 2024-05-14T16:54:09.916 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-14 16:48:20.761501877 +0000 2024-05-14T16:54:09.916 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-14 16:48:20.761501877 +0000 2024-05-14T16:54:09.916 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:54:09.979 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-14T16:54:09.979 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-14T16:54:09.979 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000180986 s, 2.8 MB/s 2024-05-14T16:54:09.980 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:54:10.036 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-14 16:53:56.538490380 +0000 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-14 16:48:21.099495591 +0000 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-14 16:48:21.099495591 +0000 2024-05-14T16:54:10.091 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-14 16:48:21.099495591 +0000 2024-05-14T16:54:10.091 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:54:10.153 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-14T16:54:10.153 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-14T16:54:10.154 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000200459 s, 2.6 MB/s 2024-05-14T16:54:10.155 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:54:10.211 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T16:54:10.211 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:54:10.228 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:54:10.228 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 16:53:46.238868696 +0000 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 16:48:32.892162990 +0000 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 16:48:32.892162990 +0000 2024-05-14T16:54:10.284 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 16:48:32.892162990 +0000 2024-05-14T16:54:10.285 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:54:10.348 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T16:54:10.348 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T16:54:10.348 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000257582 s, 2.0 MB/s 2024-05-14T16:54:10.349 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:54:10.405 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-05-14T16:54:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:10 smithi150 ceph-mon[36204]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 16:53:46.239868680 +0000 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 16:48:33.248156872 +0000 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 16:48:33.248156872 +0000 2024-05-14T16:54:10.460 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 16:48:33.248156872 +0000 2024-05-14T16:54:10.460 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:54:10.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:10 smithi099 ceph-mon[31851]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:10.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:10.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:10.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:10.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:10 smithi193 ceph-mon[35962]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:10.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:10.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:10.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:10.522 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T16:54:10.523 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T16:54:10.523 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000190894 s, 2.7 MB/s 2024-05-14T16:54:10.524 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:54:10.580 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 16:53:46.239868680 +0000 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 16:48:33.581151150 +0000 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 16:48:33.581151150 +0000 2024-05-14T16:54:10.635 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 16:48:33.581151150 +0000 2024-05-14T16:54:10.636 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:54:10.698 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T16:54:10.698 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T16:54:10.698 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000221869 s, 2.3 MB/s 2024-05-14T16:54:10.699 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:54:10.755 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-05-14T16:54:10.810 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:54:10.810 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:54:10.810 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-14T16:54:10.810 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:54:10.811 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:54:10.811 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 16:53:46.240868663 +0000 2024-05-14T16:54:10.811 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 16:48:33.910145496 +0000 2024-05-14T16:54:10.811 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 16:48:33.910145496 +0000 2024-05-14T16:54:10.811 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 16:48:33.910145496 +0000 2024-05-14T16:54:10.811 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:54:10.872 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T16:54:10.872 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T16:54:10.872 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000217724 s, 2.4 MB/s 2024-05-14T16:54:10.873 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:54:10.930 INFO:tasks.cephadm:Deploying osd.0 on smithi099 with /dev/vg_nvme/lv_4... 2024-05-14T16:54:10.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:54:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:11 smithi099 ceph-mon[31851]: Deploying daemon mgr.z on smithi193 2024-05-14T16:54:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:11.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:11.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:11.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:54:11.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:11.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:11.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:11 smithi150 ceph-mon[36204]: Deploying daemon mgr.z on smithi193 2024-05-14T16:54:11.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:11.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:11 smithi193 ceph-mon[35962]: Deploying daemon mgr.z on smithi193 2024-05-14T16:54:12.265 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 systemd[1]: Starting Ceph mgr.z for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:54:12.283 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:12 smithi099 ceph-mon[31851]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:12.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:12 smithi193 ceph-mon[35962]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:12.543 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 podman[37793]: 2024-05-14 16:54:12.354860038 +0000 UTC m=+0.093915143 container create b02a4c2b190912f25da7ab386f87baa4b2006e3fc1471b085378f8dc2fa31b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-14T16:54:12.543 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 podman[37793]: 2024-05-14 16:54:12.279423549 +0000 UTC m=+0.018478625 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:54:12.556 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:12 smithi150 ceph-mon[36204]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:12.824 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:54:12.824 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:54:12.824 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-14T16:54:12.824 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-14T16:54:12.824 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114676 s, 914 MB/s 2024-05-14T16:54:12.824 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-14T16:54:12.901 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 podman[37793]: 2024-05-14 16:54:12.562080615 +0000 UTC m=+0.301135682 container init b02a4c2b190912f25da7ab386f87baa4b2006e3fc1471b085378f8dc2fa31b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-14T16:54:12.901 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 podman[37793]: 2024-05-14 16:54:12.564768022 +0000 UTC m=+0.303823085 container start b02a4c2b190912f25da7ab386f87baa4b2006e3fc1471b085378f8dc2fa31b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:54:12.901 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 ceph-mgr[37812]: -- 172.21.15.193:0/1293596084 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599edb225a0 con 0x5599edaf7000 2024-05-14T16:54:12.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 bash[37793]: b02a4c2b190912f25da7ab386f87baa4b2006e3fc1471b085378f8dc2fa31b44 2024-05-14T16:54:12.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 systemd[1]: Started Ceph mgr.z for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:54:12.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:12.717+0000 7f97ed806200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:54:12.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:12 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:12.902+0000 7f97ed806200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:13.292 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:13.005+0000 7f97ed806200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:54:13.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-05-14T16:54:13.610 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:13.710 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:13.312+0000 7f97ed806200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:54:13.710 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:13.397+0000 7f97ed806200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:13 smithi150 ceph-mon[36204]: Deploying daemon mgr.y on smithi150 2024-05-14T16:54:14.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:14.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:14.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:14.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:14.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:14.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:54:14.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:14.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:14.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:13 smithi099 ceph-mon[31851]: Deploying daemon mgr.y on smithi150 2024-05-14T16:54:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:54:14.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:14.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:14.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:13 smithi193 ceph-mon[35962]: Deploying daemon mgr.y on smithi150 2024-05-14T16:54:14.929 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:14.580+0000 7f97ed806200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:54:14.929 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:14.751+0000 7f97ed806200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:14.929 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:14.929+0000 7f97ed806200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:54:14.945 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:14 smithi150 systemd[1]: Starting Ceph mgr.y for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:54:15.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 -- 172.21.15.99:0/1434423827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028101650 msgr2=0x7f0028101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:15.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 --2- 172.21.15.99:0/1434423827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028101650 0x7f0028101ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00180099c0 tx=0x7f0018009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:15.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 -- 172.21.15.99:0/1434423827 shutdown_connections 2024-05-14T16:54:15.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 --2- 172.21.15.99:0/1434423827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0028102010 0x7f002810bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:15.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 --2- 172.21.15.99:0/1434423827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0028104bd0 0x7f0028104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:15.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 --2- 172.21.15.99:0/1434423827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028101650 0x7f0028101ad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 -- 172.21.15.99:0/1434423827 >> 172.21.15.99:0/1434423827 conn(0x7f00280fd1f0 msgr2=0x7f00280ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 -- 172.21.15.99:0/1434423827 shutdown_connections 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 -- 172.21.15.99:0/1434423827 wait complete. 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 Processor -- start 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.097+0000 7f002f797700 1 -- start start 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0028101650 0x7f00281996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 0x7f0028199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0028104bd0 0x7f00281a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f002810f090 con 0x7f0028102010 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f002810ef10 con 0x7f0028104bd0 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f002810f210 con 0x7f0028101650 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 0x7f0028199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002d533700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0028101650 0x7f00281996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 0x7f0028199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49938/0 (socket says 172.21.15.99:49938) 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 -- 172.21.15.99:0/1564531474 learned_addr learned my addr 172.21.15.99:0/1564531474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002dd34700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0028104bd0 0x7f00281a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 -- 172.21.15.99:0/1564531474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0028101650 msgr2=0x7f00281996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0028101650 0x7f00281996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 -- 172.21.15.99:0/1564531474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0028104bd0 msgr2=0x7f00281a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0028104bd0 0x7f00281a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 -- 172.21.15.99:0/1564531474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00281a1300 con 0x7f0028102010 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002cd32700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 0x7f0028199be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0018003f10 tx=0x7f0018003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001800bea0 con 0x7f0028102010 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f001800f460 con 0x7f0028102010 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00281a1590 con 0x7f0028102010 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018017650 con 0x7f0028102010 2024-05-14T16:54:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.098+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00281a1ac0 con 0x7f0028102010 2024-05-14T16:54:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.099+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f00180178b0 con 0x7f0028102010 2024-05-14T16:54:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.099+0000 7f001e7fc700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f001403e750 0x7f0014040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.099+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f002806a230 con 0x7f0028102010 2024-05-14T16:54:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.100+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0018053780 con 0x7f0028102010 2024-05-14T16:54:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.100+0000 7f002d533700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f001403e750 0x7f0014040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.100+0000 7f002d533700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f001403e750 0x7f0014040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f002806b9b0 tx=0x7f0024009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:15.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.105+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0018013070 con 0x7f0028102010 2024-05-14T16:54:15.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:14 smithi150 podman[37568]: 2024-05-14 16:54:14.848324453 +0000 UTC m=+0.016831147 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:54:15.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:14 smithi150 podman[37568]: 2024-05-14 16:54:14.956560251 +0000 UTC m=+0.125066945 container create 6b35e6d20c5642d2fb53228bddfb5abd71dc4740d4d00ccc724707884a839a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-14T16:54:15.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 podman[37568]: 2024-05-14 16:54:15.133856723 +0000 UTC m=+0.302363411 container init 6b35e6d20c5642d2fb53228bddfb5abd71dc4740d4d00ccc724707884a839a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T16:54:15.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 podman[37568]: 2024-05-14 16:54:15.13657396 +0000 UTC m=+0.305080654 container start 6b35e6d20c5642d2fb53228bddfb5abd71dc4740d4d00ccc724707884a839a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:54:15.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mgr[37588]: -- 172.21.15.150:0/3544284849 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561dbc4445a0 con 0x561dbc419000 2024-05-14T16:54:15.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:15.319+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0028063980 con 0x7f001403e750 2024-05-14T16:54:15.476 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 bash[37568]: 6b35e6d20c5642d2fb53228bddfb5abd71dc4740d4d00ccc724707884a839a15 2024-05-14T16:54:15.477 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 systemd[1]: Started Ceph mgr.y for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:54:15.477 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:15.290+0000 7f1874b4e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:54:15.477 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:15.477+0000 7f1874b4e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:15.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:15.233+0000 7f97ed806200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:54:15.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:15.330+0000 7f97ed806200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:15.542+0000 7f97ed806200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:15.758+0000 7f97ed806200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:54:15.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:54:15.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:15.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:15.907 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:15.583+0000 7f1874b4e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:54:15.907 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:15.907+0000 7f1874b4e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:54:15.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:15.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:16.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:16.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:16.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:16.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:16.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:16.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:54:16.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:54:16.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:16.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:16.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:16 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:16.000+0000 7f1874b4e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:54:16.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:16.412+0000 7f97ed806200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:54:17.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:16 smithi099 ceph-mon[31851]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:17.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:16 smithi193 ceph-mon[35962]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:17.042 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:16.637+0000 7f97ed806200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:54:17.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:16 smithi150 ceph-mon[36204]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:17.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:17.076+0000 7f97ed806200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:54:17.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:17.172+0000 7f97ed806200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:54:17.570 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:17.226+0000 7f1874b4e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:54:17.570 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:17.394+0000 7f1874b4e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:17.570 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:17.570+0000 7f1874b4e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:17.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:17.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: Reconfiguring daemon mgr.x on smithi099 2024-05-14T16:54:17.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:17.545+0000 7f97ed806200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:17.902+0000 7f97ed806200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: Reconfiguring daemon mgr.x on smithi099 2024-05-14T16:54:17.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: Reconfiguring daemon mgr.x on smithi099 2024-05-14T16:54:18.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:18.161 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:17.859+0000 7f1874b4e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:54:18.161 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:17.953+0000 7f1874b4e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:54:18.161 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:18.161+0000 7f1874b4e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:54:18.239 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:17.996+0000 7f97ed806200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:54:18.453 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:18.352+0000 7f1874b4e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:18.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:18.240+0000 7f97ed806200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:54:18.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:18.373+0000 7f97ed806200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:54:18.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:18.469+0000 7f97ed806200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:18 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1358714530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0db35f1-6897-4148-8abf-3d8f5945234d"}]: dispatch 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:18 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1358714530' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0db35f1-6897-4148-8abf-3d8f5945234d"}]': finished 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:18 smithi193 ceph-mon[35962]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:18 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2575544582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:18.799+0000 7f97ed806200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:19.043 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:18.889+0000 7f97ed806200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:54:19.169 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:18.953+0000 7f1874b4e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:54:19.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:18 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1358714530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0db35f1-6897-4148-8abf-3d8f5945234d"}]: dispatch 2024-05-14T16:54:19.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:18 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1358714530' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0db35f1-6897-4148-8abf-3d8f5945234d"}]': finished 2024-05-14T16:54:19.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:18 smithi150 ceph-mon[36204]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:54:19.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:19.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:18 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2575544582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:54:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:18 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1358714530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0db35f1-6897-4148-8abf-3d8f5945234d"}]: dispatch 2024-05-14T16:54:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:18 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1358714530' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0db35f1-6897-4148-8abf-3d8f5945234d"}]': finished 2024-05-14T16:54:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:18 smithi099 ceph-mon[31851]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:54:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:18 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2575544582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:54:19.792 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:19.490+0000 7f97ed806200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:54:19.792 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 16:54:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T16:54:19.707+0000 7f97ed806200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:54:19.953 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:19 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:19.169+0000 7f1874b4e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:54:19.953 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:19 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:19.585+0000 7f1874b4e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:54:19.953 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:19 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:19.678+0000 7f1874b4e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:54:19.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:19.976+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f001802acc0 con 0x7f0028102010 2024-05-14T16:54:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:19 smithi099 ceph-mon[31851]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:19 smithi099 ceph-mon[31851]: Standby manager daemon z started 2024-05-14T16:54:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:19 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:54:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:19 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:19 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:54:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:19 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:19 smithi193 ceph-mon[35962]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:19 smithi193 ceph-mon[35962]: Standby manager daemon z started 2024-05-14T16:54:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:19 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:54:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:19 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:19 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:54:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:19 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:20.046+0000 7f1874b4e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:19 smithi150 ceph-mon[36204]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:19 smithi150 ceph-mon[36204]: Standby manager daemon z started 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:19 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:19 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:19 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:54:20.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:19 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/3284350160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:20.703 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:20.395+0000 7f1874b4e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:54:20.728 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:20.486+0000 7f1874b4e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:54:20.979 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:20.724+0000 7f1874b4e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:54:20.979 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:20.854+0000 7f1874b4e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:54:20.979 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:20.946+0000 7f1874b4e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:54:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:20 smithi099 ceph-mon[31851]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-14T16:54:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:20 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:54:21.271 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:20 smithi150 ceph-mon[36204]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-14T16:54:21.271 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:20 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:54:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:20 smithi193 ceph-mon[35962]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-14T16:54:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:20 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:54:21.703 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:21.271+0000 7f1874b4e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:21.703 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:21.362+0000 7f1874b4e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:54:22.157 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:21 smithi099 ceph-mon[31851]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:21 smithi193 ceph-mon[35962]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:21 smithi150 ceph-mon[36204]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:22.453 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:21.961+0000 7f1874b4e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:54:22.453 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 16:54:22 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T16:54:22.186+0000 7f1874b4e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:54:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:22.990+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f001800f930 con 0x7f0028102010 2024-05-14T16:54:23.122 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:22 smithi099 ceph-mon[31851]: Standby manager daemon y started 2024-05-14T16:54:23.123 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:22 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:54:23.123 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:22 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:23.123 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:22 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:54:23.123 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:22 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:22 smithi193 ceph-mon[35962]: Standby manager daemon y started 2024-05-14T16:54:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:22 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:54:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:22 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:22 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:54:23.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:22 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:22 smithi150 ceph-mon[36204]: Standby manager daemon y started 2024-05-14T16:54:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:22 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:54:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:22 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:22 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:54:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:22 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/2078680747' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T16:54:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:54:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:24.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T16:54:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:54:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:24.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:24.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:24.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:24.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:26.246 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:25 smithi099 ceph-mon[31851]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:26.247 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:54:26.247 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:25 smithi193 ceph-mon[35962]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:54:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:25 smithi150 ceph-mon[36204]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:54:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:27.118 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:26 smithi099 ceph-mon[31851]: Deploying daemon osd.0 on smithi099 2024-05-14T16:54:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:26 smithi193 ceph-mon[35962]: Deploying daemon osd.0 on smithi099 2024-05-14T16:54:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:26 smithi150 ceph-mon[36204]: Deploying daemon osd.0 on smithi099 2024-05-14T16:54:28.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:27 smithi099 ceph-mon[31851]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:28.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:27 smithi193 ceph-mon[35962]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:28.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:27 smithi150 ceph-mon[36204]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:30.260 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:29 smithi099 ceph-mon[31851]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:30.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:30 smithi193 ceph-mon[35962]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:30 smithi150 ceph-mon[36204]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:32.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:31 smithi150 ceph-mon[36204]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:32.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:31 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:32.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:31 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:32.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:31 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:31 smithi099 ceph-mon[31851]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:31 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:31 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:32.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:31 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:32.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:31 smithi193 ceph-mon[35962]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:32.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:31 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:32.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:31 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:32.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:31 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:33.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:33 smithi099 ceph-mon[31851]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.126+0000 7f001e7fc700 1 -- 172.21.15.99:0/1564531474 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0028063980 con 0x7f001403e750 2024-05-14T16:54:34.126 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 0 on host 'smithi099' 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f001403e750 msgr2=0x7f0014040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f001403e750 0x7f0014040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f002806b9b0 tx=0x7f0024009260 comp rx=0 tx=0).stop 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 msgr2=0x7f0028199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 0x7f0028199be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0018003f10 tx=0x7f0018003f40 comp rx=0 tx=0).stop 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 shutdown_connections 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f001403e750 0x7f0014040c10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0028104bd0 0x7f00281a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0028101650 0x7f00281996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 --2- 172.21.15.99:0/1564531474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102010 0x7f0028199be0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 >> 172.21.15.99:0/1564531474 conn(0x7f00280fd1f0 msgr2=0x7f0028107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 shutdown_connections 2024-05-14T16:54:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:54:34.128+0000 7f002f797700 1 -- 172.21.15.99:0/1564531474 wait complete. 2024-05-14T16:54:34.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:33 smithi150 ceph-mon[36204]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:34.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:33 smithi193 ceph-mon[35962]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:34.709 DEBUG:teuthology.orchestra.run.smithi099:osd.0> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.0.service 2024-05-14T16:54:34.712 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-05-14T16:54:34.712 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:54:34.954 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:54:35.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:35 smithi099 ceph-mon[31851]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:54:35.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.405 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:35 smithi150 ceph-mon[36204]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:54:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:35.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:35 smithi193 ceph-mon[35962]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:54:36.160 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:36 smithi099 ceph-mon[31851]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:36.160 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:36 smithi099 ceph-mon[31851]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:54:36.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:36 smithi099 ceph-mon[31851]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:54:36.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:36 smithi099 ceph-mon[31851]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:54:36.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:36 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:36.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:36 smithi150 ceph-mon[36204]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:36.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:36 smithi150 ceph-mon[36204]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:54:36.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:36 smithi150 ceph-mon[36204]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:54:36.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:36 smithi150 ceph-mon[36204]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:54:36.290 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:36 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:36.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:36 smithi193 ceph-mon[35962]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:36.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:36 smithi193 ceph-mon[35962]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:54:36.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:36 smithi193 ceph-mon[35962]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:54:36.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:36 smithi193 ceph-mon[35962]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:54:36.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:36 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:36.853 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:54:36.854 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:54:36.854 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-14T16:54:36.854 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-14T16:54:36.854 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112775 s, 930 MB/s 2024-05-14T16:54:36.854 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-14T16:54:37.417 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-05-14T16:54:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:37 smithi099 ceph-mon[31851]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:37 smithi099 ceph-mon[31851]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:54:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:37 smithi099 ceph-mon[31851]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:54:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:37 smithi099 ceph-mon[31851]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' 2024-05-14T16:54:37.515 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:54:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0[44343]: 2024-05-14T16:54:37.352+0000 7f84b6bd5700 -1 osd.0 0 waiting for initial osdmap 2024-05-14T16:54:37.515 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:54:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0[44343]: 2024-05-14T16:54:37.459+0000 7f84ae9c0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:54:37.652 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:54:37.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:37 smithi150 ceph-mon[36204]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:37.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:37 smithi150 ceph-mon[36204]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:54:37.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:37 smithi150 ceph-mon[36204]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:54:37.695 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:37.695 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:37.695 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:37 smithi150 ceph-mon[36204]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' 2024-05-14T16:54:37.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:37 smithi193 ceph-mon[35962]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:37.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:37 smithi193 ceph-mon[35962]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:54:37.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:37 smithi193 ceph-mon[35962]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:54:37.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:37.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:37.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:37 smithi193 ceph-mon[35962]: from='osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004]' entity='osd.0' 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi099 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:38.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi099 to 17088M 2024-05-14T16:54:38.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:38.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:38.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:38.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi099 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi099 to 17088M 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi099 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:38.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi099 to 17088M 2024-05-14T16:54:38.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:38.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:38.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:38.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:39.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 -- 172.21.15.150:0/2920439816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a8104040 msgr2=0x7f20a810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 --2- 172.21.15.150:0/2920439816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a8104040 0x7f20a810a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f20a000b0d0 tx=0x7f20a000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 -- 172.21.15.150:0/2920439816 shutdown_connections 2024-05-14T16:54:39.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 --2- 172.21.15.150:0/2920439816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a8104040 0x7f20a810a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 --2- 172.21.15.150:0/2920439816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a81024a0 0x7f20a81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 --2- 172.21.15.150:0/2920439816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 0x7f20a8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 -- 172.21.15.150:0/2920439816 >> 172.21.15.150:0/2920439816 conn(0x7f20a80fda40 msgr2=0x7f20a80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 -- 172.21.15.150:0/2920439816 shutdown_connections 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.195+0000 7f20af6dd700 1 -- 172.21.15.150:0/2920439816 wait complete. 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 Processor -- start 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 -- start start 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 0x7f20a81063e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 0x7f20a8104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a8104040 0x7f20a8104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810e030 con 0x7f20a8104040 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810deb0 con 0x7f20a81024a0 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20af6dd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810e1b0 con 0x7f20a81036a0 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 0x7f20a81063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20acc78700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 0x7f20a8104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 0x7f20a81063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53592/0 (socket says 172.21.15.150:53592) 2024-05-14T16:54:39.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 -- 172.21.15.150:0/1831948699 learned_addr learned my addr 172.21.15.150:0/1831948699 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20adc7a700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a8104040 0x7f20a8104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 -- 172.21.15.150:0/1831948699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 msgr2=0x7f20a8104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 0x7f20a8104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 -- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a8104040 msgr2=0x7f20a8104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a8104040 0x7f20a8104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 -- 172.21.15.150:0/1831948699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a8105890 con 0x7f20a81024a0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20acc78700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 0x7f20a8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20adc7a700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a8104040 0x7f20a8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.196+0000 7f20ad479700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 0x7f20a81063e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f209800b500 tx=0x7f209800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.197+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098015020 con 0x7f20a81024a0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.197+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2098011ce0 con 0x7f20a81024a0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.197+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a81a3920 con 0x7f20a81024a0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.197+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098010d20 con 0x7f20a81024a0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.197+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81a3f00 con 0x7f20a81024a0 2024-05-14T16:54:39.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.198+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a806a230 con 0x7f20a81024a0 2024-05-14T16:54:39.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.199+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2098018020 con 0x7f20a81024a0 2024-05-14T16:54:39.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.199+0000 7f209e7fc700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f20940b2870 0x7f20940b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.200+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f20980dfe90 con 0x7f20a81024a0 2024-05-14T16:54:39.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.200+0000 7f20acc78700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f20940b2870 0x7f20940b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.200+0000 7f20acc78700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f20940b2870 0x7f20940b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f20a4011490 tx=0x7f20a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:39.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.204+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f209801f020 con 0x7f20a81024a0 2024-05-14T16:54:39.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:39.417+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f20a80ffc40 con 0x7f20940b2870 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] boot 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:54:39.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:54:39.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] boot 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:39.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:54:39.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:54:39.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:54:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: osd.0 [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] boot 2024-05-14T16:54:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:54:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:39.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:39.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:54:39.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:54:39.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:40.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:40 smithi150 ceph-mon[36204]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:40.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:40 smithi150 ceph-mon[36204]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:54:40.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:40 smithi099 ceph-mon[31851]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:40.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:40 smithi099 ceph-mon[31851]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:54:40.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:40 smithi193 ceph-mon[35962]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:40.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:40 smithi193 ceph-mon[35962]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:54:41.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:41 smithi150 ceph-mon[36204]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:41.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:41 smithi099 ceph-mon[31851]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:41.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:41 smithi193 ceph-mon[35962]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:42.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:42 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/3723198676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]: dispatch 2024-05-14T16:54:42.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:42 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]: dispatch 2024-05-14T16:54:42.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:42 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]': finished 2024-05-14T16:54:42.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:42 smithi150 ceph-mon[36204]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:54:42.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:42 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:54:42.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:42 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/3723198676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]: dispatch 2024-05-14T16:54:42.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:42 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]: dispatch 2024-05-14T16:54:42.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:42 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]': finished 2024-05-14T16:54:42.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:42 smithi193 ceph-mon[35962]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:54:42.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:42 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:54:42.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:42 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/3723198676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]: dispatch 2024-05-14T16:54:42.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:42 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]: dispatch 2024-05-14T16:54:42.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:42 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9bc6113d-d89d-4515-a997-f001e08ff626"}]': finished 2024-05-14T16:54:42.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:42 smithi099 ceph-mon[31851]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:54:42.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:42 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:54:43.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:43 smithi150 ceph-mon[36204]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:43.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:43 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/2464196802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:54:43.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:43 smithi099 ceph-mon[31851]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:43.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:43 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/2464196802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:54:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:43 smithi193 ceph-mon[35962]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:43.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:43 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/2464196802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:54:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:45 smithi099 ceph-mon[31851]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:45.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:45 smithi193 ceph-mon[35962]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:45.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:45 smithi150 ceph-mon[36204]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:47.669 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:47 smithi150 ceph-mon[36204]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:47.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:47 smithi099 ceph-mon[31851]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:47 smithi193 ceph-mon[35962]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:49.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:49 smithi099 ceph-mon[31851]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:49.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:54:49.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:49.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:49 smithi099 ceph-mon[31851]: Deploying daemon osd.1 on smithi150 2024-05-14T16:54:49.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:49 smithi150 ceph-mon[36204]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:49.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:54:49.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:49.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:49 smithi150 ceph-mon[36204]: Deploying daemon osd.1 on smithi150 2024-05-14T16:54:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:49 smithi193 ceph-mon[35962]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:54:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:49 smithi193 ceph-mon[35962]: Deploying daemon osd.1 on smithi150 2024-05-14T16:54:51.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:51 smithi099 ceph-mon[31851]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:51.765 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:51 smithi150 ceph-mon[36204]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:51.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:51 smithi193 ceph-mon[35962]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:53.557 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:53 smithi150 ceph-mon[36204]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:53 smithi099 ceph-mon[31851]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:53.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:53 smithi193 ceph-mon[35962]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:55.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:55.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:55 smithi099 ceph-mon[31851]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:55.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:55.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:55.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:55.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:55 smithi150 ceph-mon[36204]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:55.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:55.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:55.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:55.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:55 smithi193 ceph-mon[35962]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:55.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:55.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:57.433 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:57.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:57.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.031+0000 7f209e7fc700 1 -- 172.21.15.150:0/1831948699 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f20a80ffc40 con 0x7f20940b2870 2024-05-14T16:54:58.031 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-05-14T16:54:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f20940b2870 msgr2=0x7f20940b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f20940b2870 0x7f20940b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f20a4011490 tx=0x7f20a4009260 comp rx=0 tx=0).stop 2024-05-14T16:54:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 msgr2=0x7f20a81063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 0x7f20a81063e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f209800b500 tx=0x7f209800b810 comp rx=0 tx=0).stop 2024-05-14T16:54:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 shutdown_connections 2024-05-14T16:54:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f20940b2870 0x7f20940b4d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:58.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a8104040 0x7f20a8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:58.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20a81024a0 0x7f20a81063e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:58.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 --2- 172.21.15.150:0/1831948699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a81036a0 0x7f20a8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:58.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 >> 172.21.15.150:0/1831948699 conn(0x7f20a80fda40 msgr2=0x7f20a80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:58.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 shutdown_connections 2024-05-14T16:54:58.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:54:58.034+0000 7f20af6dd700 1 -- 172.21.15.150:0/1831948699 wait complete. 2024-05-14T16:54:58.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:58 smithi099 ceph-mon[31851]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:58.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:58.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:58 smithi193 ceph-mon[35962]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:58.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:58.639 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:58 smithi150 ceph-mon[36204]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:54:58.639 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.639 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:54:58.639 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:58.665 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.1.service 2024-05-14T16:54:58.668 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-05-14T16:54:58.668 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:54:58.909 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:54:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:59 smithi193 ceph-mon[35962]: from='osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:54:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:54:59 smithi193 ceph-mon[35962]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:54:59.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:59 smithi099 ceph-mon[31851]: from='osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:54:59.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:54:59 smithi099 ceph-mon[31851]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:54:59.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:59 smithi150 ceph-mon[36204]: from='osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:54:59.578 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:54:59 smithi150 ceph-mon[36204]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:55:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:00 smithi099 ceph-mon[31851]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:55:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:00 smithi099 ceph-mon[31851]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:55:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:00 smithi099 ceph-mon[31851]: from='osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:55:00.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:00 smithi099 ceph-mon[31851]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:55:00.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:00.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:00 smithi099 ceph-mon[31851]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:55:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:00 smithi193 ceph-mon[35962]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:55:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:00 smithi193 ceph-mon[35962]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:55:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:00 smithi193 ceph-mon[35962]: from='osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:55:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:00 smithi193 ceph-mon[35962]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:55:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:00 smithi193 ceph-mon[35962]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:55:00.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:00 smithi150 ceph-mon[36204]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:55:00.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:00 smithi150 ceph-mon[36204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:55:00.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:00 smithi150 ceph-mon[36204]: from='osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:55:00.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:00 smithi150 ceph-mon[36204]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:55:00.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:00.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:00 smithi150 ceph-mon[36204]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:55:00.631 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1[40776]: 2024-05-14T16:55:00.263+0000 7f249f92a700 -1 osd.1 0 waiting for initial osdmap 2024-05-14T16:55:00.631 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1[40776]: 2024-05-14T16:55:00.267+0000 7f249b71d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:55:00.846 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:55:00.847 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:55:00.847 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T16:55:00.847 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T16:55:00.847 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011514 s, 911 MB/s 2024-05-14T16:55:00.847 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T16:55:01.361 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-05-14T16:55:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:01 smithi193 ceph-mon[35962]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:55:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:01 smithi193 ceph-mon[35962]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:55:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:01.606 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:55:01.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:01 smithi150 ceph-mon[36204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:55:01.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:01 smithi150 ceph-mon[36204]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:55:01.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:01.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:01.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:01 smithi099 ceph-mon[31851]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:55:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:01 smithi099 ceph-mon[31851]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:55:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi150 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi150 to 17088M 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] boot 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:02.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:02.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:55:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:55:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:55:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi150 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi150 to 17088M 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] boot 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:02.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:55:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:55:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:55:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi150 2024-05-14T16:55:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi150 to 17088M 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: osd.1 [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] boot 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:02.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:03.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.103+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/4273151764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4104050 msgr2=0x7f2fe410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:03.105 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.103+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/4273151764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4104050 0x7f2fe410a900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00b0a0 tx=0x7f2fdc00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.103+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/4273151764 shutdown_connections 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/4273151764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe4104050 0x7f2fe410a900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/4273151764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe41024b0 0x7f2fe41028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/4273151764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe41036b0 0x7f2fe4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/4273151764 >> 172.21.15.193:0/4273151764 conn(0x7f2fe40fda70 msgr2=0x7f2fe40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/4273151764 shutdown_connections 2024-05-14T16:55:03.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/4273151764 wait complete. 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 Processor -- start 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.104+0000 7f2fe9cf1700 1 -- start start 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe41024b0 0x7f2fe4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 0x7f2fe4199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe4104050 0x7f2fe41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410e090 con 0x7f2fe4104050 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410df10 con 0x7f2fe41024b0 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe410e210 con 0x7f2fe41036b0 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 0x7f2fe4199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 0x7f2fe4199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:40352/0 (socket says 172.21.15.193:40352) 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 -- 172.21.15.193:0/865593327 learned_addr learned my addr 172.21.15.193:0/865593327 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe3fff700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe4104050 0x7f2fe41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe37fe700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe41024b0 0x7f2fe4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:03.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 -- 172.21.15.193:0/865593327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe41024b0 msgr2=0x7f2fe4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe41024b0 0x7f2fe4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 -- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe4104050 msgr2=0x7f2fe41a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe4104050 0x7f2fe41a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 -- 172.21.15.193:0/865593327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe41a14f0 con 0x7f2fe41036b0 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe2ffd700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 0x7f2fe4199e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b560 tx=0x7f2fd800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8015020 con 0x7f2fe41036b0 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd8011ce0 con 0x7f2fe41036b0 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.105+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe41a17e0 con 0x7f2fe41036b0 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.106+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8010d40 con 0x7f2fe41036b0 2024-05-14T16:55:03.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.106+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe41a1df0 con 0x7f2fe41036b0 2024-05-14T16:55:03.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.106+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe406a230 con 0x7f2fe41036b0 2024-05-14T16:55:03.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.108+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2fd8018020 con 0x7f2fe41036b0 2024-05-14T16:55:03.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.108+0000 7f2fe0ff9700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2fcc0b2870 0x7f2fcc0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:03.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.108+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f2fd80e0580 con 0x7f2fe41036b0 2024-05-14T16:55:03.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.109+0000 7f2fe37fe700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2fcc0b2870 0x7f2fcc0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:03.109 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.109+0000 7f2fe37fe700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2fcc0b2870 0x7f2fcc0b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4103510 tx=0x7f2fd4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:03.112 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.112+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fd801f020 con 0x7f2fe41036b0 2024-05-14T16:55:03.325 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:03.325+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2fe406c3a0 con 0x7f2fcc0b2870 2024-05-14T16:55:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:03 smithi150 ceph-mon[36204]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:55:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:03 smithi150 ceph-mon[36204]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:03 smithi099 ceph-mon[31851]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:55:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:03 smithi099 ceph-mon[31851]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:03 smithi193 ceph-mon[35962]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:55:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:03 smithi193 ceph-mon[35962]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:04.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:04 smithi193 ceph-mon[35962]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:04.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:04 smithi150 ceph-mon[36204]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:04.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:04.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:04.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:04.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:04 smithi099 ceph-mon[31851]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:04.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:04.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:04.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:05.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:05 smithi193 ceph-mon[35962]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:05.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:05 smithi150 ceph-mon[36204]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:05.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:05 smithi099 ceph-mon[31851]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:06.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:06 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/737222422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]: dispatch 2024-05-14T16:55:06.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:06 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]: dispatch 2024-05-14T16:55:06.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:06 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]': finished 2024-05-14T16:55:06.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:06 smithi193 ceph-mon[35962]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:55:06.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:06 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:06 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/737222422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]: dispatch 2024-05-14T16:55:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:06 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]: dispatch 2024-05-14T16:55:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:06 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]': finished 2024-05-14T16:55:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:06 smithi150 ceph-mon[36204]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:55:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:06 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:06 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/737222422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]: dispatch 2024-05-14T16:55:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:06 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]: dispatch 2024-05-14T16:55:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:06 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef"}]': finished 2024-05-14T16:55:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:06 smithi099 ceph-mon[31851]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:55:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:06 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:07.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:07 smithi150 ceph-mon[36204]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:07.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:07 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/2074238854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:07.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:07 smithi099 ceph-mon[31851]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:07.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:07 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/2074238854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:07.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:07 smithi193 ceph-mon[35962]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:07.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:07 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/2074238854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:09.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:09 smithi150 ceph-mon[36204]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:09.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:09 smithi099 ceph-mon[31851]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:09.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:09 smithi193 ceph-mon[35962]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:11.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:11 smithi150 ceph-mon[36204]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:11.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:11 smithi099 ceph-mon[31851]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:11.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:11 smithi193 ceph-mon[35962]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:13.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:13 smithi193 ceph-mon[35962]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:13.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:55:13.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:13.694 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:13 smithi193 ceph-mon[35962]: Deploying daemon osd.2 on smithi193 2024-05-14T16:55:13.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:13 smithi150 ceph-mon[36204]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:13.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:55:13.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:13.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:13 smithi150 ceph-mon[36204]: Deploying daemon osd.2 on smithi193 2024-05-14T16:55:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:13 smithi099 ceph-mon[31851]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:55:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:13 smithi099 ceph-mon[31851]: Deploying daemon osd.2 on smithi193 2024-05-14T16:55:15.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:15 smithi193 ceph-mon[35962]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:15.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:15 smithi099 ceph-mon[31851]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:15.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:15 smithi150 ceph-mon[36204]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:17.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:17 smithi099 ceph-mon[31851]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:17.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:17 smithi193 ceph-mon[35962]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:17.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:17 smithi150 ceph-mon[36204]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:19.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:19 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:19.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:19 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:19.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:19 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:19.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:19 smithi193 ceph-mon[35962]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:19.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:19 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:19.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:19 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:19.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:19 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:19.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:19 smithi150 ceph-mon[36204]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:19.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:19 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:19.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:19 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:19.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:19 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:19.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:19 smithi099 ceph-mon[31851]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:21.790 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.790+0000 7f2fe0ff9700 1 -- 172.21.15.193:0/865593327 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fe406c3a0 con 0x7f2fcc0b2870 2024-05-14T16:55:21.790 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2fcc0b2870 msgr2=0x7f2fcc0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2fcc0b2870 0x7f2fcc0b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4103510 tx=0x7f2fd4009190 comp rx=0 tx=0).stop 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 msgr2=0x7f2fe4199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 0x7f2fe4199e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b560 tx=0x7f2fd800b870 comp rx=0 tx=0).stop 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 shutdown_connections 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2fcc0b2870 0x7f2fcc0b4d30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe4104050 0x7f2fe41a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe41024b0 0x7f2fe4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 --2- 172.21.15.193:0/865593327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe41036b0 0x7f2fe4199e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:21.793 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 >> 172.21.15.193:0/865593327 conn(0x7f2fe40fda70 msgr2=0x7f2fe406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:21.794 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 shutdown_connections 2024-05-14T16:55:21.794 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:55:21.793+0000 7f2fe9cf1700 1 -- 172.21.15.193:0/865593327 wait complete. 2024-05-14T16:55:22.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:22.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:22.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:22.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:22.409 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.2.service 2024-05-14T16:55:22.411 INFO:tasks.cephadm:Deploying osd.3 on smithi099 with /dev/vg_nvme/lv_3... 2024-05-14T16:55:22.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:55:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:22.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:22.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:23.082 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:22 smithi099 ceph-mon[31851]: from='osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:55:23.082 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:22 smithi099 ceph-mon[31851]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:55:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:22 smithi150 ceph-mon[36204]: from='osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:55:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:22 smithi150 ceph-mon[36204]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:55:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:22 smithi193 ceph-mon[35962]: from='osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:55:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:22 smithi193 ceph-mon[35962]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:55:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:23 smithi099 ceph-mon[31851]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:23 smithi099 ceph-mon[31851]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:55:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:23 smithi099 ceph-mon[31851]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:55:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:23 smithi099 ceph-mon[31851]: from='osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:55:24.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:23 smithi099 ceph-mon[31851]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:55:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:23 smithi193 ceph-mon[35962]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:23 smithi193 ceph-mon[35962]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:55:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:23 smithi193 ceph-mon[35962]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:55:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:23 smithi193 ceph-mon[35962]: from='osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:55:24.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:23 smithi193 ceph-mon[35962]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:55:24.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:23 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2[41088]: 2024-05-14T16:55:23.997+0000 7f17cb6ff700 -1 osd.2 0 waiting for initial osdmap 2024-05-14T16:55:24.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:24 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2[41088]: 2024-05-14T16:55:24.004+0000 7f17c74f2700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:55:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:55:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:55:24.433 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-14T16:55:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-14T16:55:24.434 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011312 s, 927 MB/s 2024-05-14T16:55:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-14T16:55:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:23 smithi150 ceph-mon[36204]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:23 smithi150 ceph-mon[36204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:55:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:23 smithi150 ceph-mon[36204]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:55:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:23 smithi150 ceph-mon[36204]: from='osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:55:24.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:23 smithi150 ceph-mon[36204]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:55:24.969 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-05-14T16:55:25.230 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:55:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:55:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:25.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:25.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:25.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:55:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:55:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:25.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:25.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:25.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:25.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:26.252 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:55:26.252 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:55:26.252 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:26.252 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi193 2024-05-14T16:55:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi193 to 17088M 2024-05-14T16:55:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] boot 2024-05-14T16:55:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:55:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:55:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:55:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi193 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi193 to 17088M 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] boot 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:55:26.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi193 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi193 to 17088M 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:26.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: osd.2 [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] boot 2024-05-14T16:55:26.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:55:26.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 -- 172.21.15.99:0/1911720021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 msgr2=0x7fcd30103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 --2- 172.21.15.99:0/1911720021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30103b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd180099c0 tx=0x7fcd18009cd0 comp rx=0 tx=0).stop 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 -- 172.21.15.99:0/1911720021 shutdown_connections 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 --2- 172.21.15.99:0/1911720021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 0x7fcd3010a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 --2- 172.21.15.99:0/1911720021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd301024a0 0x7fcd301028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 --2- 172.21.15.99:0/1911720021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30103b00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 -- 172.21.15.99:0/1911720021 >> 172.21.15.99:0/1911720021 conn(0x7fcd300fda40 msgr2=0x7fcd300ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 -- 172.21.15.99:0/1911720021 shutdown_connections 2024-05-14T16:55:26.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.749+0000 7fcd368fd700 1 -- 172.21.15.99:0/1911720021 wait complete. 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 Processor -- start 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 -- start start 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd301024a0 0x7fcd30199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 0x7fcd301a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3010df40 con 0x7fcd301036a0 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3010ddc0 con 0x7fcd30104040 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd368fd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3010e0c0 con 0x7fcd301024a0 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42634/0 (socket says 172.21.15.99:42634) 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 -- 172.21.15.99:0/268786429 learned_addr learned my addr 172.21.15.99:0/268786429 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd34e9a700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 0x7fcd301a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd2ffff700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd301024a0 0x7fcd30199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:26.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 -- 172.21.15.99:0/268786429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd301024a0 msgr2=0x7fcd30199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd301024a0 0x7fcd30199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 -- 172.21.15.99:0/268786429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 msgr2=0x7fcd301a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 0x7fcd301a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd27fff700 1 -- 172.21.15.99:0/268786429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd301a1390 con 0x7fcd301036a0 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.750+0000 7fcd34e9a700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 0x7fcd301a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.751+0000 7fcd27fff700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30199cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcd18005910 tx=0x7fcd1800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.751+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18009e60 con 0x7fcd301036a0 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.751+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301a1620 con 0x7fcd301036a0 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.751+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd18003d70 con 0x7fcd301036a0 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.751+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd301a1ad0 con 0x7fcd301036a0 2024-05-14T16:55:26.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.751+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18022420 con 0x7fcd301036a0 2024-05-14T16:55:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.752+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3006a230 con 0x7fcd301036a0 2024-05-14T16:55:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.753+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fcd180038c0 con 0x7fcd301036a0 2024-05-14T16:55:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.754+0000 7fcd2dffb700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd1c0b27f0 0x7fcd1c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.754+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fcd180e1e40 con 0x7fcd301036a0 2024-05-14T16:55:26.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.754+0000 7fcd2ffff700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd1c0b27f0 0x7fcd1c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.754+0000 7fcd2ffff700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd1c0b27f0 0x7fcd1c0b4cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd30103500 tx=0x7fcd20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:26.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.758+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd18013070 con 0x7fcd301036a0 2024-05-14T16:55:26.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:26.971+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcd300ffc60 con 0x7fcd1c0b27f0 2024-05-14T16:55:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:27 smithi099 ceph-mon[31851]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:55:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:55:27.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:27.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:27.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:27 smithi193 ceph-mon[35962]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:55:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:55:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:27 smithi150 ceph-mon[36204]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:55:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:55:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:28.209 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 ceph-mon[31851]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:28.209 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 ceph-mon[31851]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:28.209 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:55:28.209 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 ceph-mon[31851]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:55:28.209 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49145]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49145]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49145]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49148]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49148]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49148]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.210 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49148]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 ceph-mon[35962]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:28.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 ceph-mon[35962]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:28.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:55:28.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 ceph-mon[35962]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:55:28.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45111]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45111]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45114]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45114]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45114]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.293 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45114]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.348 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 ceph-mon[36204]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:28.348 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 ceph-mon[36204]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:28.348 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:55:28.348 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 ceph-mon[36204]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:55:28.348 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:55:28.348 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44464]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:55:28.348 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.349 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44464]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.349 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44464]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.349 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44467]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:55:28.349 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44467]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.349 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44467]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.349 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44467]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.493 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49153]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49156]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:28 smithi099 sudo[49156]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44470]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:55:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44470]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44470]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44470]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44473]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:55:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44473]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:28 smithi150 sudo[44473]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:55:28.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45117]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45117]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:28.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45120]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:55:28.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45120]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:55:28.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45120]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:55:28.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:28 smithi193 sudo[45120]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:55:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:29.272+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcd1800fd10 con 0x7fcd301036a0 2024-05-14T16:55:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:55:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:29 smithi193 ceph-mon[35962]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:55:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:55:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:29 smithi150 ceph-mon[36204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:55:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:55:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:55:29.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:29 smithi099 ceph-mon[31851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:55:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T16:55:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:55:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/568801158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8e6b76-114d-4622-8e95-ea570fb85003"}]: dispatch 2024-05-14T16:55:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/568801158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8e6b76-114d-4622-8e95-ea570fb85003"}]': finished 2024-05-14T16:55:30.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:55:30.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:30 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:30.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:30.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T16:55:30.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:55:30.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/568801158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8e6b76-114d-4622-8e95-ea570fb85003"}]: dispatch 2024-05-14T16:55:30.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/568801158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8e6b76-114d-4622-8e95-ea570fb85003"}]': finished 2024-05-14T16:55:30.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:55:30.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:30 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/568801158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8e6b76-114d-4622-8e95-ea570fb85003"}]: dispatch 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/568801158' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8e6b76-114d-4622-8e95-ea570fb85003"}]': finished 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:55:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:30 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:31 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3982128337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:31.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:31 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3982128337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:31 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3982128337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:32.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:32 smithi193 ceph-mon[35962]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:32.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:32 smithi150 ceph-mon[36204]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:32.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:32 smithi099 ceph-mon[31851]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:34.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:34 smithi099 ceph-mon[31851]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:34.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:34 smithi193 ceph-mon[35962]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:34.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:34 smithi150 ceph-mon[36204]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:36.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:36 smithi099 ceph-mon[31851]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:36.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:36 smithi150 ceph-mon[36204]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:36.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:36 smithi193 ceph-mon[35962]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:55:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:37.507 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:55:37.507 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:55:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:38.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:38 smithi099 ceph-mon[31851]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:38.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:38 smithi099 ceph-mon[31851]: Deploying daemon osd.3 on smithi099 2024-05-14T16:55:38.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:38 smithi150 ceph-mon[36204]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:38.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:38 smithi150 ceph-mon[36204]: Deploying daemon osd.3 on smithi099 2024-05-14T16:55:38.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:38 smithi193 ceph-mon[35962]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:38.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:38 smithi193 ceph-mon[35962]: Deploying daemon osd.3 on smithi099 2024-05-14T16:55:40.214 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:40 smithi099 ceph-mon[31851]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:40.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:40 smithi150 ceph-mon[36204]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:40.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:40 smithi193 ceph-mon[35962]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:42 smithi099 ceph-mon[31851]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:42.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:42 smithi193 ceph-mon[35962]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:42.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:42 smithi150 ceph-mon[36204]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:44.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:44 smithi099 ceph-mon[31851]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:44.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:44 smithi150 ceph-mon[36204]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:44.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:44 smithi193 ceph-mon[35962]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:45.176 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 3 on host 'smithi099' 2024-05-14T16:55:45.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.176+0000 7fcd2dffb700 1 -- 172.21.15.99:0/268786429 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcd300ffc60 con 0x7fcd1c0b27f0 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd1c0b27f0 msgr2=0x7fcd1c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd1c0b27f0 0x7fcd1c0b4cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd30103500 tx=0x7fcd20009260 comp rx=0 tx=0).stop 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 msgr2=0x7fcd30199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30199cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcd18005910 tx=0x7fcd1800f740 comp rx=0 tx=0).stop 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 shutdown_connections 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd1c0b27f0 0x7fcd1c0b4cb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:45.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd30104040 0x7fcd301a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:45.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd301024a0 0x7fcd30199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:45.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 --2- 172.21.15.99:0/268786429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd301036a0 0x7fcd30199cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:45.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 >> 172.21.15.99:0/268786429 conn(0x7fcd300fda40 msgr2=0x7fcd300ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:45.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 shutdown_connections 2024-05-14T16:55:45.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:55:45.179+0000 7fcd368fd700 1 -- 172.21.15.99:0/268786429 wait complete. 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:45.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:45.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:45.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:45.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:45.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.832 DEBUG:teuthology.orchestra.run.smithi099:osd.3> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.3.service 2024-05-14T16:55:45.834 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-05-14T16:55:45.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:55:45.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:45.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:45.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:45.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:45.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:45.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:46.080 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:55:46.118 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 16:55:45 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T16:55:45.974+0000 7fdb4b8967c0 -1 osd.3 0 log_to_monitors true 2024-05-14T16:55:46.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:46 smithi099 ceph-mon[31851]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:55:46.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:46 smithi193 ceph-mon[35962]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:55:46.865 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:46 smithi150 ceph-mon[36204]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:55:47.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:47 smithi099 ceph-mon[31851]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:47.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:47 smithi099 ceph-mon[31851]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:55:47.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:47 smithi099 ceph-mon[31851]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:55:47.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:47 smithi099 ceph-mon[31851]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:55:47.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:47 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:47.765 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 16:55:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T16:55:47.529+0000 7fdb40efa700 -1 osd.3 0 waiting for initial osdmap 2024-05-14T16:55:47.765 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 16:55:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T16:55:47.534+0000 7fdb3cced700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:55:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:47 smithi193 ceph-mon[35962]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:47 smithi193 ceph-mon[35962]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:55:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:47 smithi193 ceph-mon[35962]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:55:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:47 smithi193 ceph-mon[35962]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:55:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:47 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:47.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:47 smithi150 ceph-mon[36204]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:47.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:47 smithi150 ceph-mon[36204]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:55:47.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:47 smithi150 ceph-mon[36204]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:55:47.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:47 smithi150 ceph-mon[36204]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:55:47.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:47 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:47.955 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:55:47.955 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:55:47.955 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-14T16:55:47.955 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-14T16:55:47.955 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126905 s, 826 MB/s 2024-05-14T16:55:47.955 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-14T16:55:48.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:55:48.576 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T16:55:48.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:48.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:48.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:48.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:48.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:48.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:48.578 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-05-14T16:55:48.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:55:48.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T16:55:48.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:48.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:48.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:48.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:48.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:48.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:48.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:55:48.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T16:55:48.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:48.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:48.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:48.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:48.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:48.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:55:48.857 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:55:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi099 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi099 to 8544M 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] boot 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:49.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:49.796 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi099 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi099 to 8544M 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] boot 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:49.797 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:50.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi099 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi099 to 8544M 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: osd.3 [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] boot 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:55:50.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:55:50.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 -- 172.21.15.150:0/2245010633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 msgr2=0x7fc56810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:50.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 --2- 172.21.15.150:0/2245010633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc56810ba20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc56000b0d0 tx=0x7fc56000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:55:50.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 -- 172.21.15.150:0/2245010633 shutdown_connections 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 --2- 172.21.15.150:0/2245010633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc56810ba20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 --2- 172.21.15.150:0/2245010633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 0x7fc5681039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 --2- 172.21.15.150:0/2245010633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5681047d0 0x7fc568104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 -- 172.21.15.150:0/2245010633 >> 172.21.15.150:0/2245010633 conn(0x7fc5680feb90 msgr2=0x7fc568100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 -- 172.21.15.150:0/2245010633 shutdown_connections 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.288+0000 7fc56fef8700 1 -- 172.21.15.150:0/2245010633 wait complete. 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 Processor -- start 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 -- start start 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 0x7fc568199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5681047d0 0x7fc568199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc5681a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:50.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc56810f070 con 0x7fc5681047d0 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc56810eef0 con 0x7fc568105170 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56fef8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc56810f1f0 con 0x7fc5681035d0 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc5681a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc5681a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37108/0 (socket says 172.21.15.150:37108) 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 -- 172.21.15.150:0/264542655 learned_addr learned my addr 172.21.15.150:0/264542655 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56dc94700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 0x7fc568199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56d493700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5681047d0 0x7fc568199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 -- 172.21.15.150:0/264542655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 msgr2=0x7fc568199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 0x7fc568199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 -- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5681047d0 msgr2=0x7fc568199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5681047d0 0x7fc568199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56e495700 1 -- 172.21.15.150:0/264542655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5681a1260 con 0x7fc568105170 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.289+0000 7fc56dc94700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 0x7fc568199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:50.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.290+0000 7fc56e495700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc5681a0bc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc560009cd0 tx=0x7fc560004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.290+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001f030 con 0x7fc568105170 2024-05-14T16:55:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.290+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc560005040 con 0x7fc568105170 2024-05-14T16:55:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.290+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5681a14f0 con 0x7fc568105170 2024-05-14T16:55:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.290+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001b690 con 0x7fc568105170 2024-05-14T16:55:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.290+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5681a19f0 con 0x7fc568105170 2024-05-14T16:55:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.291+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56806a230 con 0x7fc568105170 2024-05-14T16:55:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.292+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc560022030 con 0x7fc568105170 2024-05-14T16:55:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.293+0000 7fc55effd700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc5540b2900 0x7fc5540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.293+0000 7fc56dc94700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc5540b2900 0x7fc5540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.293+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fc56000f020 con 0x7fc568105170 2024-05-14T16:55:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.293+0000 7fc56dc94700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc5540b2900 0x7fc5540b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc568104630 tx=0x7fc558009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:50.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.298+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc560029020 con 0x7fc568105170 2024-05-14T16:55:50.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:55:50.508+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc568063980 con 0x7fc5540b2900 2024-05-14T16:55:50.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:50 smithi150 ceph-mon[36204]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:55:50.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:50.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:50.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:51.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:50 smithi099 ceph-mon[31851]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:55:51.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:51.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:51.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:51.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:50 smithi193 ceph-mon[35962]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:55:51.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:55:51.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:55:51.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:51.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:51 smithi150 ceph-mon[36204]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:55:51.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:51 smithi150 ceph-mon[36204]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:51.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:51 smithi150 ceph-mon[36204]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T16:55:52.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:51 smithi099 ceph-mon[31851]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:55:52.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:51 smithi099 ceph-mon[31851]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:52.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:51 smithi099 ceph-mon[31851]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T16:55:52.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:51 smithi193 ceph-mon[35962]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:55:52.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:51 smithi193 ceph-mon[35962]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:55:52.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:51 smithi193 ceph-mon[35962]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T16:55:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:53 smithi150 ceph-mon[36204]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-14T16:55:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:53 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/1424963519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]: dispatch 2024-05-14T16:55:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:53 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]: dispatch 2024-05-14T16:55:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:53 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]': finished 2024-05-14T16:55:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:53 smithi150 ceph-mon[36204]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:55:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:53 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:55:54.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:53 smithi099 ceph-mon[31851]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-14T16:55:54.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:53 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/1424963519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]: dispatch 2024-05-14T16:55:54.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:53 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]: dispatch 2024-05-14T16:55:54.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:53 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]': finished 2024-05-14T16:55:54.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:53 smithi099 ceph-mon[31851]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:55:54.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:55:54.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:53 smithi193 ceph-mon[35962]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-14T16:55:54.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:53 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/1424963519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]: dispatch 2024-05-14T16:55:54.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:53 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]: dispatch 2024-05-14T16:55:54.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:53 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf410b8e-faab-433d-8af0-db09e90b91b9"}]': finished 2024-05-14T16:55:54.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:53 smithi193 ceph-mon[35962]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:55:54.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:55:54.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:54 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/105330750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:55.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:54 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/105330750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:55.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:54 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/105330750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:55:55.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:55 smithi150 ceph-mon[36204]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-14T16:55:56.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:55 smithi099 ceph-mon[31851]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-14T16:55:56.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:55 smithi193 ceph-mon[35962]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-14T16:55:57.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:57 smithi150 ceph-mon[36204]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-14T16:55:57.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:57 smithi099 ceph-mon[31851]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-14T16:55:58.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:57 smithi193 ceph-mon[35962]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-14T16:55:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:55:59 smithi150 ceph-mon[36204]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:56:00.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:55:59 smithi099 ceph-mon[31851]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:56:00.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:55:59 smithi193 ceph-mon[35962]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:56:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:56:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:01.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:56:01.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:01.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:56:01.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:01.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:01 smithi150 ceph-mon[36204]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:56:01.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:01 smithi150 ceph-mon[36204]: Deploying daemon osd.4 on smithi150 2024-05-14T16:56:02.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:01 smithi099 ceph-mon[31851]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:56:02.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:01 smithi099 ceph-mon[31851]: Deploying daemon osd.4 on smithi150 2024-05-14T16:56:02.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:01 smithi193 ceph-mon[35962]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:56:02.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:01 smithi193 ceph-mon[35962]: Deploying daemon osd.4 on smithi150 2024-05-14T16:56:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:03 smithi150 ceph-mon[36204]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:04.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:03 smithi099 ceph-mon[31851]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:04.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:03 smithi193 ceph-mon[35962]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:05 smithi150 ceph-mon[36204]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:06.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:05 smithi099 ceph-mon[31851]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:06.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:05 smithi193 ceph-mon[35962]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:07.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:06 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:07.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:06 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:07.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:06 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:07.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:06 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:07.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:06 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:07.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:06 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:06 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:06 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:06 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:07.926 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:07 smithi150 ceph-mon[36204]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:08.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:07 smithi099 ceph-mon[31851]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:08.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:07 smithi193 ceph-mon[35962]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.374+0000 7fc55effd700 1 -- 172.21.15.150:0/264542655 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc568063980 con 0x7fc5540b2900 2024-05-14T16:56:09.374 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-05-14T16:56:09.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc5540b2900 msgr2=0x7fc5540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc5540b2900 0x7fc5540b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc568104630 tx=0x7fc558009260 comp rx=0 tx=0).stop 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 msgr2=0x7fc5681a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc5681a0bc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc560009cd0 tx=0x7fc560004ce0 comp rx=0 tx=0).stop 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 shutdown_connections 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc5540b2900 0x7fc5540b4dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc568105170 0x7fc5681a0bc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5681035d0 0x7fc568199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 --2- 172.21.15.150:0/264542655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5681047d0 0x7fc568199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 >> 172.21.15.150:0/264542655 conn(0x7fc5680feb90 msgr2=0x7fc568107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 shutdown_connections 2024-05-14T16:56:09.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:56:09.377+0000 7fc56fef8700 1 -- 172.21.15.150:0/264542655 wait complete. 2024-05-14T16:56:09.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:09.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:09.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:09.982 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.4.service 2024-05-14T16:56:09.984 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-05-14T16:56:09.984 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:10.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:10.227 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:56:10.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:10 smithi150 ceph-mon[36204]: from='osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:56:10.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:10 smithi150 ceph-mon[36204]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:56:11.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:10 smithi099 ceph-mon[31851]: from='osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:56:11.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:10 smithi099 ceph-mon[31851]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:56:11.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:10 smithi193 ceph-mon[35962]: from='osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:56:11.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:10 smithi193 ceph-mon[35962]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:56:11.729 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:11 smithi193 ceph-mon[35962]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:11.729 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:11 smithi193 ceph-mon[35962]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:56:11.729 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:11 smithi193 ceph-mon[35962]: from='osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:56:11.729 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:11 smithi193 ceph-mon[35962]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:56:11.729 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:11.729 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:11 smithi193 ceph-mon[35962]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:56:11.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:11 smithi150 ceph-mon[36204]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:11.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:11 smithi150 ceph-mon[36204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:56:11.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:11 smithi150 ceph-mon[36204]: from='osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:56:11.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:11 smithi150 ceph-mon[36204]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:56:11.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:11.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:11 smithi150 ceph-mon[36204]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:56:11.954 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 16:56:11 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4[47246]: 2024-05-14T16:56:11.634+0000 7f783fb8a700 -1 osd.4 0 waiting for initial osdmap 2024-05-14T16:56:11.954 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 16:56:11 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4[47246]: 2024-05-14T16:56:11.641+0000 7f783b97d700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:56:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:11 smithi099 ceph-mon[31851]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:11 smithi099 ceph-mon[31851]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:56:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:11 smithi099 ceph-mon[31851]: from='osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:56:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:11 smithi099 ceph-mon[31851]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:56:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:12.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:11 smithi099 ceph-mon[31851]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:56:12.071 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:56:12.071 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:56:12.071 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T16:56:12.071 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T16:56:12.071 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113903 s, 921 MB/s 2024-05-14T16:56:12.072 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T16:56:12.756 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-05-14T16:56:12.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:12 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:56:12.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:12 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:56:12.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:12 smithi150 ceph-mon[36204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:56:12.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:12 smithi150 ceph-mon[36204]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T16:56:12.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:12 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:12.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:12 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:12 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:56:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:12 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:56:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:12 smithi099 ceph-mon[31851]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:56:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:12 smithi099 ceph-mon[31851]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T16:56:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:12 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:12 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.035 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:56:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:12 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:56:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:12 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:56:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:12 smithi193 ceph-mon[35962]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:56:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:12 smithi193 ceph-mon[35962]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T16:56:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:12 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:12 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:13.744 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi150 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] boot 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:13.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi150 2024-05-14T16:56:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] boot 2024-05-14T16:56:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:13.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:14.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:56:14.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi150 2024-05-14T16:56:14.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: osd.4 [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] boot 2024-05-14T16:56:14.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:56:14.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:14.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:14.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 -- 172.21.15.193:0/1581878730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a801035b0 msgr2=0x7f4a801039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:14.515 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 --2- 172.21.15.193:0/1581878730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a801035b0 0x7f4a801039d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b0a0 tx=0x7f4a7c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:56:14.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 -- 172.21.15.193:0/1581878730 shutdown_connections 2024-05-14T16:56:14.560 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 --2- 172.21.15.193:0/1581878730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80105150 0x7f4a8010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:14.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 --2- 172.21.15.193:0/1581878730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a801035b0 0x7f4a801039d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:14.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 --2- 172.21.15.193:0/1581878730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a801047b0 0x7f4a80104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:14.614 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 -- 172.21.15.193:0/1581878730 >> 172.21.15.193:0/1581878730 conn(0x7f4a800feb50 msgr2=0x7f4a80100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:14.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 -- 172.21.15.193:0/1581878730 shutdown_connections 2024-05-14T16:56:14.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.493+0000 7f4a875e3700 1 -- 172.21.15.193:0/1581878730 wait complete. 2024-05-14T16:56:14.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 Processor -- start 2024-05-14T16:56:14.790 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 -- start start 2024-05-14T16:56:14.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a801035b0 0x7f4a8019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:14.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a801047b0 0x7f4a8019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:14.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 0x7f4a801a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:14.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8010f020 con 0x7f4a801035b0 2024-05-14T16:56:14.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8010eea0 con 0x7f4a801047b0 2024-05-14T16:56:14.874 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a875e3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8010f1a0 con 0x7f4a80105150 2024-05-14T16:56:14.932 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 0x7f4a801a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:14.939 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 0x7f4a801a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53556/0 (socket says 172.21.15.193:53556) 2024-05-14T16:56:14.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 -- 172.21.15.193:0/308081079 learned_addr learned my addr 172.21.15.193:0/308081079 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:56:14.953 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a8537f700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a801035b0 0x7f4a8019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:14.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a84b7e700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a801047b0 0x7f4a8019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:14.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 -- 172.21.15.193:0/308081079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a801047b0 msgr2=0x7f4a8019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:14.974 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a801047b0 0x7f4a8019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:14.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 -- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a801035b0 msgr2=0x7f4a8019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:14.988 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a801035b0 0x7f4a8019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:14.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 -- 172.21.15.193:0/308081079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a801a3500 con 0x7f4a80105150 2024-05-14T16:56:15.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.494+0000 7f4a85b80700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 0x7f4a801a2e60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c930 tx=0x7f4a7800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:15.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.495+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a780078d0 con 0x7f4a80105150 2024-05-14T16:56:15.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.495+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a801a37f0 con 0x7f4a80105150 2024-05-14T16:56:15.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.495+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7800f450 con 0x7f4a80105150 2024-05-14T16:56:15.030 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.495+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78018640 con 0x7f4a80105150 2024-05-14T16:56:15.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.495+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801a3d30 con 0x7f4a80105150 2024-05-14T16:56:15.045 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:14 smithi099 ceph-mon[31851]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T16:56:15.061 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:14 smithi150 ceph-mon[36204]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T16:56:15.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.496+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8006a230 con 0x7f4a80105150 2024-05-14T16:56:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.497+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4a78007a70 con 0x7f4a80105150 2024-05-14T16:56:15.086 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.497+0000 7f4a767fc700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4a6c0b2880 0x7f4a6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:15.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.497+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f4a780e1000 con 0x7f4a80105150 2024-05-14T16:56:15.099 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.498+0000 7f4a8537f700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4a6c0b2880 0x7f4a6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:15.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.498+0000 7f4a8537f700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4a6c0b2880 0x7f4a6c0b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009fd0 tx=0x7f4a7c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:15.113 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.502+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a78014030 con 0x7f4a80105150 2024-05-14T16:56:15.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:14.718+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4a80076e90 con 0x7f4a6c0b2880 2024-05-14T16:56:15.126 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:14 smithi193 ceph-mon[35962]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T16:56:15.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:15 smithi150 ceph-mon[36204]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:15.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:15 smithi150 ceph-mon[36204]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:56:15.968 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:56:15.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:56:15.982 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:15 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:16.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:15 smithi099 ceph-mon[31851]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:16.022 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:15 smithi099 ceph-mon[31851]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:56:16.100 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:56:16.107 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:56:16.114 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:15 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:16.122 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:15 smithi193 ceph-mon[35962]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:16.129 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:15 smithi193 ceph-mon[35962]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:56:16.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:56:16.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:56:16.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:15 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:17.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:17 smithi150 ceph-mon[36204]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:17.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:17 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/3570827204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]: dispatch 2024-05-14T16:56:17.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:17 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]: dispatch 2024-05-14T16:56:17.988 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:17 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]': finished 2024-05-14T16:56:17.995 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:17 smithi150 ceph-mon[36204]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:56:18.001 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:18.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:17 smithi099 ceph-mon[31851]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:18.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:17 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/3570827204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]: dispatch 2024-05-14T16:56:18.030 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:17 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]: dispatch 2024-05-14T16:56:18.094 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:17 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]': finished 2024-05-14T16:56:18.100 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:17 smithi099 ceph-mon[31851]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:56:18.140 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:18.147 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:17 smithi193 ceph-mon[35962]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:18.154 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:17 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/3570827204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]: dispatch 2024-05-14T16:56:18.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:17 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]: dispatch 2024-05-14T16:56:18.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:17 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd"}]': finished 2024-05-14T16:56:18.174 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:17 smithi193 ceph-mon[35962]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:56:18.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:18.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:18 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/3944616953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:56:19.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:18 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/3944616953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:56:19.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:18 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/3944616953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:56:19.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:19 smithi150 ceph-mon[36204]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:20.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:19 smithi099 ceph-mon[31851]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:20.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:19 smithi193 ceph-mon[35962]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:21.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:21 smithi193 ceph-mon[35962]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:21.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:21 smithi150 ceph-mon[36204]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:22.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:21 smithi099 ceph-mon[31851]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:23.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:23 smithi150 ceph-mon[36204]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:23.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:23 smithi193 ceph-mon[35962]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:24.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:23 smithi099 ceph-mon[31851]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:24.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:24 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:56:25.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:24 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:25.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:24 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:56:25.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:24 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:25.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:56:25.686 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:25.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:25 smithi150 ceph-mon[36204]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:26.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:25 smithi150 ceph-mon[36204]: Deploying daemon osd.5 on smithi193 2024-05-14T16:56:26.032 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:25 smithi099 ceph-mon[31851]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:26.039 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:25 smithi099 ceph-mon[31851]: Deploying daemon osd.5 on smithi193 2024-05-14T16:56:26.046 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:25 smithi193 ceph-mon[35962]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:26.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:25 smithi193 ceph-mon[35962]: Deploying daemon osd.5 on smithi193 2024-05-14T16:56:27.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:27 smithi150 ceph-mon[36204]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:28.018 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:27 smithi099 ceph-mon[31851]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:28.024 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:27 smithi193 ceph-mon[35962]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:29.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:29 smithi150 ceph-mon[36204]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:30.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:29 smithi099 ceph-mon[31851]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:30.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:29 smithi193 ceph-mon[35962]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:31.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:30 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:31.222 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:30 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:31.228 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:30 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:31.260 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:30 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:31.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:30 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:31.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:30 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:31.306 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:30 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:31.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:30 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:31.323 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:30 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:32.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:31 smithi150 ceph-mon[36204]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:31 smithi099 ceph-mon[31851]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:32.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:31 smithi193 ceph-mon[35962]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:33.085 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-05-14T16:56:33.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.085+0000 7f4a767fc700 1 -- 172.21.15.193:0/308081079 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a80076e90 con 0x7f4a6c0b2880 2024-05-14T16:56:33.101 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4a6c0b2880 msgr2=0x7f4a6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:33.108 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4a6c0b2880 0x7f4a6c0b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009fd0 tx=0x7f4a7c00b7b0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.114 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 msgr2=0x7f4a801a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:33.120 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 0x7f4a801a2e60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c930 tx=0x7f4a7800cc40 comp rx=0 tx=0).stop 2024-05-14T16:56:33.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 shutdown_connections 2024-05-14T16:56:33.214 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4a6c0b2880 0x7f4a6c0b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.220 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a80105150 0x7f4a801a2e60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.227 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a801035b0 0x7f4a8019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.235 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 --2- 172.21.15.193:0/308081079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a801047b0 0x7f4a8019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:33.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 >> 172.21.15.193:0/308081079 conn(0x7f4a800feb50 msgr2=0x7f4a80075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:33.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 shutdown_connections 2024-05-14T16:56:33.254 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:56:33.088+0000 7f4a875e3700 1 -- 172.21.15.193:0/308081079 wait complete. 2024-05-14T16:56:33.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:33.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.686 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:33.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:33.753 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.767 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.773 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:33.818 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:33.824 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.836 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:33.843 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:33.851 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.858 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.865 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.871 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:33.876 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.887 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:33.927 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:33.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.940 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.946 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:33.987 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:33.997 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.5.service 2024-05-14T16:56:34.007 INFO:tasks.cephadm:Deploying osd.6 on smithi099 with /dev/vg_nvme/lv_2... 2024-05-14T16:56:34.014 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:56:34.268 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:34.691 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:34 smithi099 ceph-mon[31851]: from='osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:56:34.699 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:34 smithi099 ceph-mon[31851]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:56:34.706 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:34 smithi150 ceph-mon[36204]: from='osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:56:34.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:34 smithi150 ceph-mon[36204]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:56:34.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:34 smithi193 ceph-mon[35962]: from='osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:56:34.736 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:34 smithi193 ceph-mon[35962]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:56:35.583 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:35 smithi099 ceph-mon[31851]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:35.588 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:35 smithi099 ceph-mon[31851]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:56:35.595 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:35 smithi099 ceph-mon[31851]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:56:35.600 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:35.605 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:35 smithi099 ceph-mon[31851]: from='osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:56:35.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:35 smithi099 ceph-mon[31851]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:56:35.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:35 smithi193 ceph-mon[35962]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:35.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:35 smithi193 ceph-mon[35962]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:56:35.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:35 smithi193 ceph-mon[35962]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:56:35.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:35.821 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:35 smithi193 ceph-mon[35962]: from='osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:56:35.828 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:35 smithi193 ceph-mon[35962]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:56:35.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:35 smithi150 ceph-mon[36204]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:35.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:35 smithi150 ceph-mon[36204]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:56:35.966 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:35 smithi150 ceph-mon[36204]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:56:35.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:35.980 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:35 smithi150 ceph-mon[36204]: from='osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:56:35.987 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:35 smithi150 ceph-mon[36204]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:56:36.137 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:56:36.144 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:56:36.150 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-14T16:56:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-14T16:56:36.162 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011438 s, 917 MB/s 2024-05-14T16:56:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-14T16:56:36.454 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 16:56:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T16:56:36.358+0000 7fa198ab3700 -1 osd.5 0 waiting for initial osdmap 2024-05-14T16:56:36.677 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:36 smithi099 ceph-mon[31851]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:56:36.684 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:36 smithi099 ceph-mon[31851]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T16:56:36.690 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:36 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:36.735 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:36 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:36.741 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:36 smithi099 ceph-mon[31851]: from='osd.5 ' entity='osd.5' 2024-05-14T16:56:36.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-05-14T16:56:36.792 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 16:56:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T16:56:36.497+0000 7fa19009d700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:56:36.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:36 smithi193 ceph-mon[35962]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:56:36.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:36 smithi193 ceph-mon[35962]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T16:56:36.812 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:36 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:36.837 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:36 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:36.843 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:36 smithi193 ceph-mon[35962]: from='osd.5 ' entity='osd.5' 2024-05-14T16:56:36.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:36 smithi150 ceph-mon[36204]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:56:36.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:36 smithi150 ceph-mon[36204]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T16:56:36.966 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:36 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:37.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:36 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:37.035 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:36 smithi150 ceph-mon[36204]: from='osd.5 ' entity='osd.5' 2024-05-14T16:56:37.046 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:37.675 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:56:37.686 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:56:37.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:37.698 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:37.705 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: osdmap e37: 6 total, 5 up, 6 in 2024-05-14T16:56:37.711 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:37.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi193 2024-05-14T16:56:37.721 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:37.729 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:37.770 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:37.776 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:37.781 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi193 to 8544M 2024-05-14T16:56:37.787 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:37.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:37.845 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:37.852 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:37.858 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:56:37.864 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:56:37.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:37.948 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:37.955 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: osdmap e37: 6 total, 5 up, 6 in 2024-05-14T16:56:37.964 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:37.971 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi193 2024-05-14T16:56:37.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:37.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:37.989 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:38.012 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:38.018 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi193 to 8544M 2024-05-14T16:56:38.024 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:38.030 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:38.035 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:38.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:38.091 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:56:38.097 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:56:38.103 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:38.137 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:38.144 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: osdmap e37: 6 total, 5 up, 6 in 2024-05-14T16:56:38.151 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:38.157 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi193 2024-05-14T16:56:38.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:38.169 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:38.175 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:38.180 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:56:38.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi193 to 8544M 2024-05-14T16:56:38.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:38.207 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:38.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:38.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:38.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 -- 172.21.15.99:0/1146266337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 msgr2=0x7fd64010b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 --2- 172.21.15.99:0/1146266337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd64010b6f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd63800b0a0 tx=0x7fd63800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 -- 172.21.15.99:0/1146266337 shutdown_connections 2024-05-14T16:56:38.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 --2- 172.21.15.99:0/1146266337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd64010b6f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 --2- 172.21.15.99:0/1146266337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd640104080 0x7fd640106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 --2- 172.21.15.99:0/1146266337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6401069b0 0x7fd640108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.451+0000 7fd646695700 1 -- 172.21.15.99:0/1146266337 >> 172.21.15.99:0/1146266337 conn(0x7fd6400fda30 msgr2=0x7fd6400ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:38.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.452+0000 7fd646695700 1 -- 172.21.15.99:0/1146266337 shutdown_connections 2024-05-14T16:56:38.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.452+0000 7fd646695700 1 -- 172.21.15.99:0/1146266337 wait complete. 2024-05-14T16:56:38.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.452+0000 7fd646695700 1 Processor -- start 2024-05-14T16:56:38.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.452+0000 7fd646695700 1 -- start start 2024-05-14T16:56:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd640104080 0x7fd6401995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:38.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6401069b0 0x7fd640199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:38.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd6401a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:38.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010eff0 con 0x7fd6401092e0 2024-05-14T16:56:38.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010ee70 con 0x7fd640104080 2024-05-14T16:56:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010f170 con 0x7fd6401069b0 2024-05-14T16:56:38.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd6401a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:38.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd6401a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60150/0 (socket says 172.21.15.99:60150) 2024-05-14T16:56:38.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 -- 172.21.15.99:0/717586634 learned_addr learned my addr 172.21.15.99:0/717586634 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:56:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd63ffff700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd640104080 0x7fd6401995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:38.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd63f7fe700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6401069b0 0x7fd640199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 -- 172.21.15.99:0/717586634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6401069b0 msgr2=0x7fd640199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:38.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6401069b0 0x7fd640199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 -- 172.21.15.99:0/717586634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd640104080 msgr2=0x7fd6401995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:38.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd640104080 0x7fd6401995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:38.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 -- 172.21.15.99:0/717586634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6401a1240 con 0x7fd6401092e0 2024-05-14T16:56:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd644c32700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd6401a0ba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd63800ced0 tx=0x7fd63800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638021070 con 0x7fd6401092e0 2024-05-14T16:56:38.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd638007d00 con 0x7fd6401092e0 2024-05-14T16:56:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6401a14d0 con 0x7fd6401092e0 2024-05-14T16:56:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.453+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63801d450 con 0x7fd6401092e0 2024-05-14T16:56:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.454+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6401a19c0 con 0x7fd6401092e0 2024-05-14T16:56:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.454+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64006a230 con 0x7fd6401092e0 2024-05-14T16:56:38.885 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:38.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:38 smithi193 ceph-mon[35962]: osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] boot 2024-05-14T16:56:38.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:38 smithi193 ceph-mon[35962]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:56:38.938 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:38.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.456+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd638004000 con 0x7fd6401092e0 2024-05-14T16:56:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.457+0000 7fd63d7fa700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fd6280b2900 0x7fd6280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:38.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.457+0000 7fd63ffff700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fd6280b2900 0x7fd6280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:38.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.457+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4566+0+0 (secure 0 0 0) 0x7fd6380e28f0 con 0x7fd6401092e0 2024-05-14T16:56:38.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.457+0000 7fd63ffff700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fd6280b2900 0x7fd6280b4dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd63000ac50 tx=0x7fd63000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:39.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.460+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd6380ab4a0 con 0x7fd6401092e0 2024-05-14T16:56:39.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:39.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:38 smithi099 ceph-mon[31851]: osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] boot 2024-05-14T16:56:39.029 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:38 smithi099 ceph-mon[31851]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:56:39.036 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:39.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:38.675+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd640063980 con 0x7fd6280b2900 2024-05-14T16:56:39.050 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:39.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:38 smithi150 ceph-mon[36204]: osd.5 [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] boot 2024-05-14T16:56:39.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:38 smithi150 ceph-mon[36204]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:56:39.142 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:56:39.533 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:39 smithi099 ceph-mon[31851]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:39.580 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:39 smithi099 ceph-mon[31851]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T16:56:39.586 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:39 smithi099 ceph-mon[31851]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:56:39.594 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:56:39.602 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:56:39.609 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:39.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:39 smithi193 ceph-mon[35962]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:39.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:39 smithi193 ceph-mon[35962]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T16:56:39.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:39 smithi193 ceph-mon[35962]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:56:39.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:56:39.855 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:56:39.864 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:39.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:39 smithi150 ceph-mon[36204]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:56:39.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:39 smithi150 ceph-mon[36204]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T16:56:39.967 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:39 smithi150 ceph-mon[36204]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:56:40.018 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:56:40.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:56:40.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:40.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:40 smithi099 ceph-mon[31851]: osdmap e40: 6 total, 6 up, 6 in 2024-05-14T16:56:40.824 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:40 smithi193 ceph-mon[35962]: osdmap e40: 6 total, 6 up, 6 in 2024-05-14T16:56:40.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:40 smithi150 ceph-mon[36204]: osdmap e40: 6 total, 6 up, 6 in 2024-05-14T16:56:41.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:41 smithi099 ceph-mon[31851]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:41.807 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:41 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1510544837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a01d0f1-49fc-4260-92d1-e75e3ec36b14"}]: dispatch 2024-05-14T16:56:41.814 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:41 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1510544837' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a01d0f1-49fc-4260-92d1-e75e3ec36b14"}]': finished 2024-05-14T16:56:41.820 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:41 smithi099 ceph-mon[31851]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:56:41.827 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:56:41.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:41 smithi193 ceph-mon[35962]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:41.907 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:41 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1510544837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a01d0f1-49fc-4260-92d1-e75e3ec36b14"}]: dispatch 2024-05-14T16:56:41.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:41 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1510544837' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a01d0f1-49fc-4260-92d1-e75e3ec36b14"}]': finished 2024-05-14T16:56:41.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:41 smithi193 ceph-mon[35962]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:56:41.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:41 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:56:41.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:41 smithi150 ceph-mon[36204]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:42.051 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:41 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1510544837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a01d0f1-49fc-4260-92d1-e75e3ec36b14"}]: dispatch 2024-05-14T16:56:42.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:41 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1510544837' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a01d0f1-49fc-4260-92d1-e75e3ec36b14"}]': finished 2024-05-14T16:56:42.063 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:41 smithi150 ceph-mon[36204]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:56:42.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:41 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:56:42.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:42 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1346823356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:56:42.826 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:42 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1346823356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:56:42.854 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:42 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1346823356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:56:43.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:43 smithi099 ceph-mon[31851]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-14T16:56:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:43 smithi193 ceph-mon[35962]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-14T16:56:43.866 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:43 smithi150 ceph-mon[36204]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-14T16:56:45.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:45 smithi099 ceph-mon[31851]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:56:45.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:45 smithi193 ceph-mon[35962]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:56:45.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:45 smithi150 ceph-mon[36204]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:56:47.726 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:47 smithi099 ceph-mon[31851]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-14T16:56:47.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:47 smithi193 ceph-mon[35962]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-14T16:56:47.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:47 smithi150 ceph-mon[36204]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-14T16:56:48.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:56:48.833 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:48.873 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:56:48.879 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:48.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:56:48.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:49.762 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:49 smithi099 ceph-mon[31851]: Deploying daemon osd.6 on smithi099 2024-05-14T16:56:49.771 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:49 smithi099 ceph-mon[31851]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T16:56:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:49 smithi193 ceph-mon[35962]: Deploying daemon osd.6 on smithi099 2024-05-14T16:56:49.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:49 smithi193 ceph-mon[35962]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T16:56:49.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:49 smithi150 ceph-mon[36204]: Deploying daemon osd.6 on smithi099 2024-05-14T16:56:49.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:49 smithi150 ceph-mon[36204]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T16:56:51.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:51 smithi099 ceph-mon[31851]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:56:51.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:51 smithi193 ceph-mon[35962]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:56:51.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:51 smithi150 ceph-mon[36204]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:56:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:53 smithi099 ceph-mon[31851]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-14T16:56:53.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:53 smithi193 ceph-mon[35962]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-14T16:56:53.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:53 smithi150 ceph-mon[36204]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-14T16:56:54.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:54 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:54.781 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:54 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:54.788 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:54 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:54.795 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:54 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:54.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:54 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:54.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:54 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:54.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:54 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:54.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:54 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:54.966 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:54 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:55.763 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:55 smithi099 ceph-mon[31851]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:55.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:55 smithi193 ceph-mon[35962]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:55.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:55 smithi150 ceph-mon[36204]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:57.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:56 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.022 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:56 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.029 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:56 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:57.035 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:56 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:57.041 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:56 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.127 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 6 on host 'smithi099' 2024-05-14T16:56:57.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.126+0000 7fd63d7fa700 1 -- 172.21.15.99:0/717586634 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd640063980 con 0x7fd6280b2900 2024-05-14T16:56:57.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.128+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fd6280b2900 msgr2=0x7fd6280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:57.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.128+0000 7fd646695700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fd6280b2900 0x7fd6280b4dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd63000ac50 tx=0x7fd63000a380 comp rx=0 tx=0).stop 2024-05-14T16:56:57.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.128+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 msgr2=0x7fd6401a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:57.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.128+0000 7fd646695700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd6401a0ba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd63800ced0 tx=0x7fd63800cf00 comp rx=0 tx=0).stop 2024-05-14T16:56:57.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 shutdown_connections 2024-05-14T16:56:57.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fd6280b2900 0x7fd6280b4dc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6401092e0 0x7fd6401a0ba0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd640104080 0x7fd6401995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 --2- 172.21.15.99:0/717586634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6401069b0 0x7fd640199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 >> 172.21.15.99:0/717586634 conn(0x7fd6400fda30 msgr2=0x7fd6401049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:57.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 shutdown_connections 2024-05-14T16:56:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:56:57.129+0000 7fd646695700 1 -- 172.21.15.99:0/717586634 wait complete. 2024-05-14T16:56:57.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:56 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:56 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.387 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:56 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:57.393 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:56 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:57.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:56 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:56 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:56 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.459 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:56 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:57.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:56 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:57.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:56 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:57.701 DEBUG:teuthology.orchestra.run.smithi099:osd.6> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.6.service 2024-05-14T16:56:57.710 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-05-14T16:56:57.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:56:57.969 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:56:58.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:58 smithi099 ceph-mon[31851]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:58.294 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:58.316 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:58.323 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:58.330 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:58 smithi099 ceph-mon[31851]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:56:58.338 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:58 smithi150 ceph-mon[36204]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:58.346 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:58.388 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:58.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:58.404 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:58 smithi150 ceph-mon[36204]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:56:58.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:58 smithi193 ceph-mon[35962]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:56:58.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:58.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:56:58.577 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:58.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:58 smithi193 ceph-mon[35962]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:56:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:59 smithi099 ceph-mon[31851]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:56:59.298 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:59 smithi099 ceph-mon[31851]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T16:56:59.306 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:59 smithi099 ceph-mon[31851]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:56:59.332 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:56:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:56:59.434 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:59 smithi150 ceph-mon[36204]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:56:59.439 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:59 smithi150 ceph-mon[36204]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T16:56:59.446 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:59 smithi150 ceph-mon[36204]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:56:59.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:56:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:56:59.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:59 smithi193 ceph-mon[35962]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:56:59.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:59 smithi193 ceph-mon[35962]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T16:56:59.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:59 smithi193 ceph-mon[35962]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:56:59.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:56:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.206 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:57:00.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:57:00.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: osdmap e43: 7 total, 6 up, 7 in 2024-05-14T16:57:00.315 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.322 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.328 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' 2024-05-14T16:57:00.334 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.340 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 16:57:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T16:57:00.014+0000 7f0b0ad40700 -1 osd.6 0 waiting for initial osdmap 2024-05-14T16:57:00.374 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 16:57:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T16:57:00.108+0000 7f0b02b2b700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:57:00.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:57:00.387 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:57:00.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: osdmap e43: 7 total, 6 up, 7 in 2024-05-14T16:57:00.417 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.422 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.444 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' 2024-05-14T16:57:00.450 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.458 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:57:00.465 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:57:00.513 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-14T16:57:00.519 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-14T16:57:00.536 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112763 s, 930 MB/s 2024-05-14T16:57:00.542 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-14T16:57:00.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:57:00.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:57:00.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: osdmap e43: 7 total, 6 up, 7 in 2024-05-14T16:57:00.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: from='osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310]' entity='osd.6' 2024-05-14T16:57:00.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:00.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-05-14T16:57:01.086 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:57:01.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:57:01.403 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:57:01.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.416 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.506 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.512 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.518 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:01.524 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:01.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.572 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] boot 2024-05-14T16:57:01.579 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T16:57:01.585 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:01.591 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:57:01.598 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:57:01.604 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.677 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:01.765 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:01.773 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] boot 2024-05-14T16:57:01.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T16:57:01.811 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:01.833 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:57:01.840 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:57:01.847 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.853 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.860 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.866 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.898 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:01.904 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.911 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:01.919 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:01.949 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:01.956 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: osd.6 [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] boot 2024-05-14T16:57:01.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T16:57:01.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:57:02.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:02 smithi150 ceph-mon[36204]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:57:02.379 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:02 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi099 2024-05-14T16:57:02.386 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:02 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi099 to 5696M 2024-05-14T16:57:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:02 smithi099 ceph-mon[31851]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:57:02.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:02 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi099 2024-05-14T16:57:02.528 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:02 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi099 to 5696M 2024-05-14T16:57:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:02 smithi193 ceph-mon[35962]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:57:02.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:02 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi099 2024-05-14T16:57:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:02 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi099 to 5696M 2024-05-14T16:57:02.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.573+0000 7f60b46af700 1 -- 172.21.15.150:0/3323571252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1069e0 msgr2=0x7f60ac108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:02.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.573+0000 7f60b46af700 1 --2- 172.21.15.150:0/3323571252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1069e0 0x7f60ac108dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f609c009990 tx=0x7f609c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- 172.21.15.150:0/3323571252 shutdown_connections 2024-05-14T16:57:02.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 --2- 172.21.15.150:0/3323571252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 0x7f60ac10b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 --2- 172.21.15.150:0/3323571252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ac1040b0 0x7f60ac1064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 --2- 172.21.15.150:0/3323571252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1069e0 0x7f60ac108dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- 172.21.15.150:0/3323571252 >> 172.21.15.150:0/3323571252 conn(0x7f60ac0fda40 msgr2=0x7f60ac0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- 172.21.15.150:0/3323571252 shutdown_connections 2024-05-14T16:57:02.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- 172.21.15.150:0/3323571252 wait complete. 2024-05-14T16:57:02.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 Processor -- start 2024-05-14T16:57:02.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- start start 2024-05-14T16:57:02.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 0x7f60ac199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:02.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ac1069e0 0x7f60ac199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:02.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 0x7f60ac1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:02.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10f0b0 con 0x7f60ac109310 2024-05-14T16:57:02.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10ef30 con 0x7f60ac1040b0 2024-05-14T16:57:02.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.574+0000 7f60b46af700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10f230 con 0x7f60ac1069e0 2024-05-14T16:57:02.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 0x7f60ac199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:02.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 0x7f60ac199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60034/0 (socket says 172.21.15.150:60034) 2024-05-14T16:57:02.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 -- 172.21.15.150:0/2945338551 learned_addr learned my addr 172.21.15.150:0/2945338551 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-14T16:57:02.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b2c4c700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 0x7f60ac1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:02.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b1c4a700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ac1069e0 0x7f60ac199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:02.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 -- 172.21.15.150:0/2945338551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ac1069e0 msgr2=0x7f60ac199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:02.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ac1069e0 0x7f60ac199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 -- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 msgr2=0x7f60ac1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:02.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 0x7f60ac1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 -- 172.21.15.150:0/2945338551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac1a13d0 con 0x7f60ac1040b0 2024-05-14T16:57:02.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b2c4c700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 0x7f60ac1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:03.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b244b700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 0x7f60ac199770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d980 tx=0x7f60a800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:03.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8017070 con 0x7f60ac1040b0 2024-05-14T16:57:03.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a8004830 con 0x7f60ac1040b0 2024-05-14T16:57:03.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1a16c0 con 0x7f60ac1040b0 2024-05-14T16:57:03.034 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005440 con 0x7f60ac1040b0 2024-05-14T16:57:03.042 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.575+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a1bb0 con 0x7f60ac1040b0 2024-05-14T16:57:03.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.576+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac06a230 con 0x7f60ac1040b0 2024-05-14T16:57:03.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.578+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f60a80055e0 con 0x7f60ac1040b0 2024-05-14T16:57:03.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.578+0000 7f60a37fe700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f60980b2880 0x7f60980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.579+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f60a80e18a0 con 0x7f60ac1040b0 2024-05-14T16:57:03.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.579+0000 7f60b1c4a700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f60980b2880 0x7f60980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.579+0000 7f60b1c4a700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f60980b2880 0x7f60980b4d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098e0 tx=0x7f609c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:03.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.582+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60a8020020 con 0x7f60ac1040b0 2024-05-14T16:57:03.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:02.793+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f60ac063980 con 0x7f60980b2880 2024-05-14T16:57:03.395 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:03 smithi150 ceph-mon[36204]: osdmap e45: 7 total, 7 up, 7 in 2024-05-14T16:57:03.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:03.416 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:03.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:03 smithi099 ceph-mon[31851]: osdmap e45: 7 total, 7 up, 7 in 2024-05-14T16:57:03.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:03.533 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:03.542 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:03.548 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:03 smithi193 ceph-mon[35962]: osdmap e45: 7 total, 7 up, 7 in 2024-05-14T16:57:03.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:03.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:03.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:04.259 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:04 smithi150 ceph-mon[36204]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:04.268 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:04 smithi150 ceph-mon[36204]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:04.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:04 smithi099 ceph-mon[31851]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:04.523 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:04 smithi099 ceph-mon[31851]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:04.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:04 smithi193 ceph-mon[35962]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:04.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:04 smithi193 ceph-mon[35962]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:06.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/3989870901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]: dispatch 2024-05-14T16:57:06.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]: dispatch 2024-05-14T16:57:06.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]': finished 2024-05-14T16:57:06.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:57:06.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:06.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:06 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/2715057553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:06.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:06.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/3989870901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]: dispatch 2024-05-14T16:57:06.735 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]: dispatch 2024-05-14T16:57:06.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]': finished 2024-05-14T16:57:06.750 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:57:06.773 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:06.781 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:06 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/2715057553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:06.870 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:06.958 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/3989870901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]: dispatch 2024-05-14T16:57:06.964 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]: dispatch 2024-05-14T16:57:06.971 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77c663e2-65e8-46c2-9363-bf3fecf22dd9"}]': finished 2024-05-14T16:57:06.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:57:07.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:07.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:06 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/2715057553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:08.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:08 smithi150 ceph-mon[36204]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:08.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:08 smithi099 ceph-mon[31851]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:08.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:08 smithi193 ceph-mon[35962]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:10 smithi150 ceph-mon[36204]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:10.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:10 smithi099 ceph-mon[31851]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:10.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:10 smithi193 ceph-mon[35962]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:12.389 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:12 smithi150 ceph-mon[36204]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:12.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:12 smithi099 ceph-mon[31851]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:12.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:12 smithi193 ceph-mon[35962]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:13.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:57:13.448 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:13.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:57:13.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:13.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:57:13.548 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:14.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:14 smithi150 ceph-mon[36204]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:14.459 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:14 smithi150 ceph-mon[36204]: Deploying daemon osd.7 on smithi150 2024-05-14T16:57:14.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:14 smithi099 ceph-mon[31851]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:14.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:14 smithi099 ceph-mon[31851]: Deploying daemon osd.7 on smithi150 2024-05-14T16:57:14.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:14 smithi193 ceph-mon[35962]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:14.548 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:14 smithi193 ceph-mon[35962]: Deploying daemon osd.7 on smithi150 2024-05-14T16:57:16.353 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:16 smithi150 ceph-mon[36204]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:16.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:16 smithi099 ceph-mon[31851]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:16.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:16 smithi193 ceph-mon[35962]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:18 smithi150 ceph-mon[36204]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:18.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:18.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:18.473 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:18.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:18 smithi099 ceph-mon[31851]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:18.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:18.534 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:18.541 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:18.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:18 smithi193 ceph-mon[35962]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:18.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:18.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:18.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:20.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:20 smithi150 ceph-mon[36204]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:20.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:20 smithi099 ceph-mon[31851]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:20.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:20 smithi193 ceph-mon[35962]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:21.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.656+0000 7f60a37fe700 1 -- 172.21.15.150:0/2945338551 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60ac063980 con 0x7f60980b2880 2024-05-14T16:57:21.692 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-05-14T16:57:21.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f60980b2880 msgr2=0x7f60980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:21.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f60980b2880 0x7f60980b4d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098e0 tx=0x7f609c003860 comp rx=0 tx=0).stop 2024-05-14T16:57:21.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 msgr2=0x7f60ac199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:21.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 0x7f60ac199770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d980 tx=0x7f60a800dd40 comp rx=0 tx=0).stop 2024-05-14T16:57:21.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 shutdown_connections 2024-05-14T16:57:21.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f60980b2880 0x7f60980b4d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:21.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60ac109310 0x7f60ac1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:21.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60ac1040b0 0x7f60ac199770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:21.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 --2- 172.21.15.150:0/2945338551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ac1069e0 0x7f60ac199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:21.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 >> 172.21.15.150:0/2945338551 conn(0x7f60ac0fda40 msgr2=0x7f60ac1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:21.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 shutdown_connections 2024-05-14T16:57:21.778 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:57:21.658+0000 7f60b46af700 1 -- 172.21.15.150:0/2945338551 wait complete. 2024-05-14T16:57:22.178 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.7.service 2024-05-14T16:57:22.189 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-05-14T16:57:22.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:57:22.450 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:57:22.458 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:22 smithi150 ceph-mon[36204]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:22.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:22 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:22.472 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:22 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:22.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:22 smithi150 ceph-mon[36204]: from='osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:57:22.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:22 smithi150 ceph-mon[36204]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:57:22.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:22 smithi099 ceph-mon[31851]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:22.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:22 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:22.528 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:22 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:22.550 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:22 smithi099 ceph-mon[31851]: from='osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:57:22.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:22 smithi099 ceph-mon[31851]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:57:22.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:22 smithi193 ceph-mon[35962]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:22.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:22 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:22.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:22 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:22.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:22 smithi193 ceph-mon[35962]: from='osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:57:22.646 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:22 smithi193 ceph-mon[35962]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:57:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:23.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:23.509 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:23.515 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:57:23.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: from='osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:57:23.536 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T16:57:23.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:23.548 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:23 smithi150 ceph-mon[36204]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:57:23.649 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:23.656 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:23.665 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:23.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:57:23.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: from='osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:57:23.688 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T16:57:23.695 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:23.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:23 smithi193 ceph-mon[35962]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:57:23.769 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:23.777 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:23.784 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:23.804 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:57:23.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: from='osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:57:23.817 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T16:57:23.825 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:23.832 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:23 smithi099 ceph-mon[31851]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:57:23.922 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 16:57:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T16:57:23.653+0000 7f81c3bb3700 -1 osd.7 0 waiting for initial osdmap 2024-05-14T16:57:23.930 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 16:57:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T16:57:23.667+0000 7f81bf9a6700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:57:24.358 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:57:24.367 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:57:24.374 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T16:57:24.381 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T16:57:24.387 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148894 s, 704 MB/s 2024-05-14T16:57:24.393 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T16:57:24.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:24 smithi150 ceph-mon[36204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:57:24.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:24 smithi150 ceph-mon[36204]: osdmap e48: 8 total, 7 up, 8 in 2024-05-14T16:57:24.968 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:24 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:24.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:24 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:25.013 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:24 smithi193 ceph-mon[35962]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:57:25.019 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:24 smithi193 ceph-mon[35962]: osdmap e48: 8 total, 7 up, 8 in 2024-05-14T16:57:25.027 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:24 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:25.033 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:24 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:25.039 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:24 smithi099 ceph-mon[31851]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:57:25.046 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:24 smithi099 ceph-mon[31851]: osdmap e48: 8 total, 7 up, 8 in 2024-05-14T16:57:25.053 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:25.061 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:25.072 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-05-14T16:57:25.315 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:57:25.716 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:57:25.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:57:25.730 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:25.739 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] boot 2024-05-14T16:57:25.828 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:57:25.846 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:25.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T16:57:25.884 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:25.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:25.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:25.905 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:25.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:25.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:25.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:25.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:25 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:25.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:57:25.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:57:25.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:25.967 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] boot 2024-05-14T16:57:25.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:57:26.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:26.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T16:57:26.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.042 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.049 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:26.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:26.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.074 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:26.080 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:26.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:25 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.094 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:57:26.103 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:57:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:57:26.116 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: osd.7 [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] boot 2024-05-14T16:57:26.122 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:57:26.128 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:57:26.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T16:57:26.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:26.183 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:26.188 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:26.195 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.201 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:26.209 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:26.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:25 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:26.720 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 -- 172.21.15.193:0/3940552246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8108130 msgr2=0x7f02c810a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 --2- 172.21.15.193:0/3940552246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8108130 0x7f02c810a5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b0a0 tx=0x7f02c000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 -- 172.21.15.193:0/3940552246 shutdown_connections 2024-05-14T16:57:26.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 --2- 172.21.15.193:0/3940552246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8108130 0x7f02c810a5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 --2- 172.21.15.193:0/3940552246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c8102ce0 0x7f02c81050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.757 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 --2- 172.21.15.193:0/3940552246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 0x7f02c8107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.763 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 -- 172.21.15.193:0/3940552246 >> 172.21.15.193:0/3940552246 conn(0x7f02c80fcd90 msgr2=0x7f02c80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:26.795 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.718+0000 7f02cf386700 1 -- 172.21.15.193:0/3940552246 shutdown_connections 2024-05-14T16:57:26.802 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.719+0000 7f02cf386700 1 -- 172.21.15.193:0/3940552246 wait complete. 2024-05-14T16:57:26.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.719+0000 7f02cf386700 1 Processor -- start 2024-05-14T16:57:26.832 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.719+0000 7f02cf386700 1 -- start start 2024-05-14T16:57:26.838 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.719+0000 7f02cf386700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 0x7f02c81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.719+0000 7f02cf386700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 0x7f02c8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.853 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.719+0000 7f02cf386700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c8108130 0x7f02c819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:26.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cf386700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810df90 con 0x7f02c8105610 2024-05-14T16:57:26.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cf386700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810de10 con 0x7f02c8108130 2024-05-14T16:57:26.873 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cf386700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810e110 con 0x7f02c8102ce0 2024-05-14T16:57:26.879 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 0x7f02c81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.885 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 0x7f02c81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54106/0 (socket says 172.21.15.193:54106) 2024-05-14T16:57:26.892 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 -- 172.21.15.193:0/3050176816 learned_addr learned my addr 172.21.15.193:0/3050176816 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:57:26.900 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cc921700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 0x7f02c8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.906 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 -- 172.21.15.193:0/3050176816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c8108130 msgr2=0x7f02c819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd923700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c8108130 0x7f02c819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:26.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c8108130 0x7f02c819fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.987 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 -- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 msgr2=0x7f02c8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:26.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 0x7f02c8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:27.000 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 -- 172.21.15.193:0/3050176816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c81a0200 con 0x7f02c8102ce0 2024-05-14T16:57:27.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cc921700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 0x7f02c8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:27.022 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cd122700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 0x7f02c81985a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009920 tx=0x7f02c4009ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:27.029 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400d020 con 0x7f02c8102ce0 2024-05-14T16:57:27.035 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c400d1c0 con 0x7f02c8102ce0 2024-05-14T16:57:27.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c8101a60 con 0x7f02c8102ce0 2024-05-14T16:57:27.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40076b0 con 0x7f02c8102ce0 2024-05-14T16:57:27.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.720+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c8101fd0 con 0x7f02c8102ce0 2024-05-14T16:57:27.063 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi150 2024-05-14T16:57:27.070 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-14T16:57:27.081 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: osdmap e51: 8 total, 8 up, 8 in 2024-05-14T16:57:28.032 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:28.139 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:28.198 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:28.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:28.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi150 2024-05-14T16:57:28.250 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-14T16:57:28.257 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: osdmap e51: 8 total, 8 up, 8 in 2024-05-14T16:57:28.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:28.271 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:28.278 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:28.285 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:28.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.721+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c806a230 con 0x7f02c8102ce0 2024-05-14T16:57:28.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.723+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f02c401f020 con 0x7f02c8102ce0 2024-05-14T16:57:28.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.724+0000 7f02be7fc700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f02b40b2880 0x7f02b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.354 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.724+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7f02c40e3280 con 0x7f02c8102ce0 2024-05-14T16:57:28.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.724+0000 7f02cc921700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f02b40b2880 0x7f02b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.365 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.724+0000 7f02cc921700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f02b40b2880 0x7f02b40b4d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f02b80099d0 tx=0x7f02b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:28.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.728+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02c4026030 con 0x7f02c8102ce0 2024-05-14T16:57:28.375 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:26.937+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f02c8102230 con 0x7f02b40b2880 2024-05-14T16:57:28.416 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi150 2024-05-14T16:57:28.422 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-14T16:57:28.427 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: osdmap e51: 8 total, 8 up, 8 in 2024-05-14T16:57:28.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:28.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:28.507 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:28.535 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:28.541 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:27 smithi193 ceph-mon[35962]: pgmap v173: 1 pgs: 1 peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:28.547 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:27 smithi193 ceph-mon[35962]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:28.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:28.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:28.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:27 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:28.572 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:27 smithi099 ceph-mon[31851]: pgmap v173: 1 pgs: 1 peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:28.579 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:27 smithi099 ceph-mon[31851]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:28.584 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:28.589 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:28.609 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:27 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:28.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:27 smithi150 ceph-mon[36204]: pgmap v173: 1 pgs: 1 peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:28.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:27 smithi150 ceph-mon[36204]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:28.697 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:28.716 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:28.721 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:27 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:30.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:29 smithi099 ceph-mon[31851]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:30.032 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:29 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/1298754779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]: dispatch 2024-05-14T16:57:30.039 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:29 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]: dispatch 2024-05-14T16:57:30.044 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:29 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]': finished 2024-05-14T16:57:30.052 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:29 smithi099 ceph-mon[31851]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:57:30.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:30.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:29 smithi193 ceph-mon[35962]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:30.071 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:29 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/1298754779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]: dispatch 2024-05-14T16:57:30.077 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:29 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]: dispatch 2024-05-14T16:57:30.084 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:29 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]': finished 2024-05-14T16:57:30.090 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:29 smithi193 ceph-mon[35962]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:57:30.096 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:30.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:29 smithi150 ceph-mon[36204]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:30.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:29 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/1298754779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]: dispatch 2024-05-14T16:57:30.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:29 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]: dispatch 2024-05-14T16:57:30.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:29 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e6281da-4f42-4ba0-8033-cac6194b5a55"}]': finished 2024-05-14T16:57:30.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:29 smithi150 ceph-mon[36204]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:57:30.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:31.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/1664290931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:31.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:30 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/1664290931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:31.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:30 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/1664290931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:32.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:31 smithi099 ceph-mon[31851]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:32.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:31 smithi193 ceph-mon[35962]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:32.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:31 smithi150 ceph-mon[36204]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:34.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:33 smithi099 ceph-mon[31851]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-14T16:57:34.071 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:33 smithi193 ceph-mon[35962]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-14T16:57:34.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:33 smithi150 ceph-mon[36204]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-14T16:57:35.885 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:35 smithi193 ceph-mon[35962]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-14T16:57:36.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:35 smithi099 ceph-mon[31851]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-14T16:57:36.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:35 smithi150 ceph-mon[36204]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-14T16:57:37.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:37 smithi193 ceph-mon[35962]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:57:37.869 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:57:37.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:37.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:37 smithi193 ceph-mon[35962]: Deploying daemon osd.8 on smithi193 2024-05-14T16:57:38.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:37 smithi099 ceph-mon[31851]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:57:38.022 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:57:38.029 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:38.035 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:37 smithi099 ceph-mon[31851]: Deploying daemon osd.8 on smithi193 2024-05-14T16:57:38.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:37 smithi150 ceph-mon[36204]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:57:38.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:57:38.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:38.223 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:37 smithi150 ceph-mon[36204]: Deploying daemon osd.8 on smithi193 2024-05-14T16:57:40.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:39 smithi099 ceph-mon[31851]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:57:40.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:39 smithi193 ceph-mon[35962]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:57:40.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:39 smithi150 ceph-mon[36204]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:57:41.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:41 smithi193 ceph-mon[35962]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-14T16:57:42.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:41 smithi099 ceph-mon[31851]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-14T16:57:42.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:41 smithi150 ceph-mon[36204]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-14T16:57:43.279 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:43.339 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:43.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:43.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:43.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:43.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:43.527 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:44.260 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:44 smithi193 ceph-mon[35962]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:57:44.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:44 smithi150 ceph-mon[36204]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:57:44.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:44 smithi099 ceph-mon[31851]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:57:45.700 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-05-14T16:57:45.708 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.699+0000 7f02be7fc700 1 -- 172.21.15.193:0/3050176816 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f02c8102230 con 0x7f02b40b2880 2024-05-14T16:57:45.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f02b40b2880 msgr2=0x7f02b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:45.722 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f02b40b2880 0x7f02b40b4d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f02b80099d0 tx=0x7f02b8008040 comp rx=0 tx=0).stop 2024-05-14T16:57:45.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 msgr2=0x7f02c81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:45.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 0x7f02c81985a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009920 tx=0x7f02c4009ce0 comp rx=0 tx=0).stop 2024-05-14T16:57:45.810 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 shutdown_connections 2024-05-14T16:57:45.817 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f02b40b2880 0x7f02b40b4d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:45.826 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c8108130 0x7f02c819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:45.833 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c8102ce0 0x7f02c81985a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:45.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 --2- 172.21.15.193:0/3050176816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02c8105610 0x7f02c8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:45.846 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 >> 172.21.15.193:0/3050176816 conn(0x7f02c80fcd90 msgr2=0x7f02c8103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:45.853 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 shutdown_connections 2024-05-14T16:57:45.859 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:57:45.702+0000 7f02cf386700 1 -- 172.21.15.193:0/3050176816 wait complete. 2024-05-14T16:57:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:45.992 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:46.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:46.046 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.060 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:45 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:46.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:46.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.218 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:46.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:46.245 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.253 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.269 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:45 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:46.277 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:46.284 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:46.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:46.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.370 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:46.399 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:45 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:46.407 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.8.service 2024-05-14T16:57:46.416 INFO:tasks.cephadm:Deploying osd.9 on smithi099 with /dev/vg_nvme/lv_1... 2024-05-14T16:57:46.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:57:46.681 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:47.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:46 smithi193 ceph-mon[35962]: from='osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:57:47.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:46 smithi193 ceph-mon[35962]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:57:47.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:46 smithi099 ceph-mon[31851]: from='osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:57:47.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:46 smithi099 ceph-mon[31851]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:57:47.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:46 smithi150 ceph-mon[36204]: from='osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:57:47.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:46 smithi150 ceph-mon[36204]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:57:48.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:47 smithi193 ceph-mon[35962]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:48.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:47 smithi193 ceph-mon[35962]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:57:48.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:47 smithi193 ceph-mon[35962]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T16:57:48.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:47 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:48.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:47 smithi193 ceph-mon[35962]: from='osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:57:48.079 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:47 smithi193 ceph-mon[35962]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:57:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:47 smithi099 ceph-mon[31851]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:48.126 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:47 smithi099 ceph-mon[31851]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:57:48.133 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:47 smithi099 ceph-mon[31851]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T16:57:48.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:47 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:48.187 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:47 smithi099 ceph-mon[31851]: from='osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:57:48.196 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:47 smithi099 ceph-mon[31851]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:57:48.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:47 smithi150 ceph-mon[36204]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:48.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:47 smithi150 ceph-mon[36204]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:57:48.218 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:47 smithi150 ceph-mon[36204]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T16:57:48.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:47 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:48.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:47 smithi150 ceph-mon[36204]: from='osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:57:48.342 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:47 smithi150 ceph-mon[36204]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:57:48.625 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:57:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:57:48.639 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-14T16:57:48.647 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-14T16:57:48.654 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114788 s, 913 MB/s 2024-05-14T16:57:48.698 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-14T16:57:48.860 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 16:57:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T16:57:48.758+0000 7f860cf7c700 -1 osd.8 0 waiting for initial osdmap 2024-05-14T16:57:48.916 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 16:57:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T16:57:48.845+0000 7f8604d67700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:57:49.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:48 smithi150 ceph-mon[36204]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:57:49.253 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:48 smithi150 ceph-mon[36204]: osdmap e54: 9 total, 8 up, 9 in 2024-05-14T16:57:49.259 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:49.266 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:48 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:49.272 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:48 smithi150 ceph-mon[36204]: from='osd.8 ' entity='osd.8' 2024-05-14T16:57:49.280 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:48 smithi099 ceph-mon[31851]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:57:49.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:48 smithi099 ceph-mon[31851]: osdmap e54: 9 total, 8 up, 9 in 2024-05-14T16:57:49.311 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:49.319 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:48 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:49.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:48 smithi099 ceph-mon[31851]: from='osd.8 ' entity='osd.8' 2024-05-14T16:57:49.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:48 smithi193 ceph-mon[35962]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:57:49.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:48 smithi193 ceph-mon[35962]: osdmap e54: 9 total, 8 up, 9 in 2024-05-14T16:57:49.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:49.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:48 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:49.447 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:48 smithi193 ceph-mon[35962]: from='osd.8 ' entity='osd.8' 2024-05-14T16:57:49.455 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-05-14T16:57:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:50.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:57:50.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:57:50.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:50.183 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:50.190 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] boot 2024-05-14T16:57:50.197 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:57:50.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:50.260 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.270 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.330 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.336 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.356 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:50.443 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:50.450 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:49 smithi099 ceph-mon[31851]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T16:57:50.466 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:57:50.473 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:57:50.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:50.487 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:50.494 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] boot 2024-05-14T16:57:50.500 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:57:50.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:50.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.541 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.595 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.602 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:50.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:50.650 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.656 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:49 smithi193 ceph-mon[35962]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T16:57:50.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:57:50.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:57:50.677 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:57:50.683 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:50.689 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: osd.8 [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] boot 2024-05-14T16:57:50.694 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:57:50.734 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:57:50.740 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.747 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.773 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.788 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.795 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:57:50.807 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.813 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:50.843 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:50.852 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:57:50.858 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:49 smithi150 ceph-mon[36204]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T16:57:51.101 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:50 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi193 2024-05-14T16:57:51.107 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:50 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi193 to 5696M 2024-05-14T16:57:51.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:50 smithi099 ceph-mon[31851]: osdmap e57: 9 total, 9 up, 9 in 2024-05-14T16:57:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:50 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi193 2024-05-14T16:57:51.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:50 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi193 to 5696M 2024-05-14T16:57:51.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:50 smithi150 ceph-mon[36204]: osdmap e57: 9 total, 9 up, 9 in 2024-05-14T16:57:51.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 -- 172.21.15.99:0/2005583304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8106960 msgr2=0x7fbab8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:51.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2005583304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8106960 0x7fbab8108d50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009990 tx=0x7fbaac009ca0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 -- 172.21.15.99:0/2005583304 shutdown_connections 2024-05-14T16:57:51.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2005583304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbab8109290 0x7fbab810b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2005583304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab8104030 0x7fbab8106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2005583304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8106960 0x7fbab8108d50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 -- 172.21.15.99:0/2005583304 >> 172.21.15.99:0/2005583304 conn(0x7fbab80fda40 msgr2=0x7fbab80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:51.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 -- 172.21.15.99:0/2005583304 shutdown_connections 2024-05-14T16:57:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 -- 172.21.15.99:0/2005583304 wait complete. 2024-05-14T16:57:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.205+0000 7fbabdb85700 1 Processor -- start 2024-05-14T16:57:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 -- start start 2024-05-14T16:57:51.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 0x7fbab8103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:51.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab8106960 0x7fbab8101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:51.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbab8109290 0x7fbab8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:51.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbab810f020 con 0x7fbab8104030 2024-05-14T16:57:51.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbab810eea0 con 0x7fbab8106960 2024-05-14T16:57:51.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbabdb85700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbab810f1a0 con 0x7fbab8109290 2024-05-14T16:57:51.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 0x7fbab8103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:51.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 0x7fbab8103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36374/0 (socket says 172.21.15.99:36374) 2024-05-14T16:57:51.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 -- 172.21.15.99:0/2783655773 learned_addr learned my addr 172.21.15.99:0/2783655773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:57:51.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab7fff700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbab8109290 0x7fbab8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 -- 172.21.15.99:0/2783655773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbab8109290 msgr2=0x7fbab8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:51.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbab8109290 0x7fbab8102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 -- 172.21.15.99:0/2783655773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab8106960 msgr2=0x7fbab8101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:51.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab6ffd700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab8106960 0x7fbab8101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:51.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab8106960 0x7fbab8101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 -- 172.21.15.99:0/2783655773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab8102dd0 con 0x7fbab8104030 2024-05-14T16:57:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.206+0000 7fbab77fe700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 0x7fbab8103990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80060b0 tx=0x7fbaa8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:51.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.207+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa801a070 con 0x7fbab8104030 2024-05-14T16:57:51.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.207+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab81a3890 con 0x7fbab8104030 2024-05-14T16:57:51.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.207+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa800de20 con 0x7fbab8104030 2024-05-14T16:57:51.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.207+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8014720 con 0x7fbab8104030 2024-05-14T16:57:51.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.207+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81a3dd0 con 0x7fbab8104030 2024-05-14T16:57:51.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.208+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab806a230 con 0x7fbab8104030 2024-05-14T16:57:51.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.209+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbaa801d070 con 0x7fbab8104030 2024-05-14T16:57:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.209+0000 7fbab4ff9700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbaa00b2880 0x7fbaa00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:51.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.210+0000 7fbab6ffd700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbaa00b2880 0x7fbaa00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:51.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.210+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7fbaa80e1640 con 0x7fbab8104030 2024-05-14T16:57:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.210+0000 7fbab6ffd700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbaa00b2880 0x7fbaa00b4d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006010 tx=0x7fbaac003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:51.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.213+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbaa8010070 con 0x7fbab8104030 2024-05-14T16:57:51.687 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:50 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi193 2024-05-14T16:57:51.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:50 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi193 to 5696M 2024-05-14T16:57:51.732 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:50 smithi193 ceph-mon[35962]: osdmap e57: 9 total, 9 up, 9 in 2024-05-14T16:57:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:57:51.426+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbab8063980 con 0x7fbaa00b2880 2024-05-14T16:57:52.142 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:51 smithi099 ceph-mon[31851]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:52.150 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:51 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:52.157 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:51 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:52.164 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:51 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:52.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:51 smithi150 ceph-mon[36204]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:52.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:51 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:52.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:51 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:52.273 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:51 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:52.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:51 smithi193 ceph-mon[35962]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:52.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:51 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:57:52.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:51 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:57:52.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:51 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:53.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:52 smithi193 ceph-mon[35962]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:53.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:52 smithi150 ceph-mon[36204]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:53.673 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:52 smithi099 ceph-mon[31851]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:57:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:53 smithi150 ceph-mon[36204]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:53 smithi099 ceph-mon[31851]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:53 smithi193 ceph-mon[35962]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:54 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3517735473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "302e14e8-2084-4de2-947a-3df7b3f8e45e"}]: dispatch 2024-05-14T16:57:55.288 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:54 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3517735473' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "302e14e8-2084-4de2-947a-3df7b3f8e45e"}]': finished 2024-05-14T16:57:55.294 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:54 smithi150 ceph-mon[36204]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:57:55.349 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:54 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:57:55.362 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:54 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2009849965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:55.387 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:54 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3517735473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "302e14e8-2084-4de2-947a-3df7b3f8e45e"}]: dispatch 2024-05-14T16:57:55.393 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:54 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3517735473' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "302e14e8-2084-4de2-947a-3df7b3f8e45e"}]': finished 2024-05-14T16:57:55.399 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:54 smithi193 ceph-mon[35962]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:57:55.405 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:54 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:57:55.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:54 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2009849965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:55.416 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:54 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3517735473' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "302e14e8-2084-4de2-947a-3df7b3f8e45e"}]: dispatch 2024-05-14T16:57:55.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:54 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3517735473' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "302e14e8-2084-4de2-947a-3df7b3f8e45e"}]': finished 2024-05-14T16:57:55.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:54 smithi099 ceph-mon[31851]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:57:55.543 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:54 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:57:55.555 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:54 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2009849965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:57:56.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:56 smithi150 ceph-mon[36204]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:56 smithi099 ceph-mon[31851]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:56.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:56 smithi193 ceph-mon[35962]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:57:58.260 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:57:58 smithi099 ceph-mon[31851]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-14T16:57:58.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:57:58 smithi150 ceph-mon[36204]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-14T16:57:58.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:57:58 smithi193 ceph-mon[35962]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-14T16:58:00.169 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:00 smithi099 ceph-mon[31851]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:58:00.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:00 smithi150 ceph-mon[36204]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:58:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:00 smithi193 ceph-mon[35962]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:58:02.287 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:02 smithi099 ceph-mon[31851]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:58:02.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:58:02.319 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:02.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:02 smithi150 ceph-mon[36204]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:58:02.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:58:02.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:02 smithi193 ceph-mon[35962]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:58:02.548 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:58:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:03.320 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:03 smithi099 ceph-mon[31851]: Deploying daemon osd.9 on smithi099 2024-05-14T16:58:03.454 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:03 smithi150 ceph-mon[36204]: Deploying daemon osd.9 on smithi099 2024-05-14T16:58:03.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:03 smithi193 ceph-mon[35962]: Deploying daemon osd.9 on smithi099 2024-05-14T16:58:04.212 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:04 smithi099 ceph-mon[31851]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:58:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:04 smithi150 ceph-mon[36204]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:58:04.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:04 smithi193 ceph-mon[35962]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:58:06.265 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:06 smithi099 ceph-mon[31851]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-14T16:58:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:06 smithi150 ceph-mon[36204]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-14T16:58:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:06 smithi193 ceph-mon[35962]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-14T16:58:08.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:07 smithi150 ceph-mon[36204]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:58:08.266 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:08.274 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:08.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:08.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:07 smithi099 ceph-mon[31851]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:58:08.294 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:08.300 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:08.306 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:08.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:07 smithi193 ceph-mon[35962]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:58:08.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:08.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:08.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:10.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:09 smithi099 ceph-mon[31851]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:10.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:09 smithi150 ceph-mon[36204]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:10.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:09 smithi193 ceph-mon[35962]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:10.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.750+0000 7fbab4ff9700 1 -- 172.21.15.99:0/2783655773 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbab8063980 con 0x7fbaa00b2880 2024-05-14T16:58:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 9 on host 'smithi099' 2024-05-14T16:58:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbaa00b2880 msgr2=0x7fbaa00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:10.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbaa00b2880 0x7fbaa00b4d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006010 tx=0x7fbaac003860 comp rx=0 tx=0).stop 2024-05-14T16:58:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 msgr2=0x7fbab8103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:10.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 0x7fbab8103990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80060b0 tx=0x7fbaa8007f90 comp rx=0 tx=0).stop 2024-05-14T16:58:10.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 shutdown_connections 2024-05-14T16:58:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbaa00b2880 0x7fbaa00b4d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:10.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbab8109290 0x7fbab8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:10.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab8104030 0x7fbab8103990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:10.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 --2- 172.21.15.99:0/2783655773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab8106960 0x7fbab8101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:10.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 >> 172.21.15.99:0/2783655773 conn(0x7fbab80fda40 msgr2=0x7fbab8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:10.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 shutdown_connections 2024-05-14T16:58:10.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:58:10.753+0000 7fbabdb85700 1 -- 172.21.15.99:0/2783655773 wait complete. 2024-05-14T16:58:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.229 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:11.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:11.255 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.295 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.301 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:58:11.308 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:11.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.329 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:11.335 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:11.340 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.346 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.352 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.359 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:58:11.365 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:11.419 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.426 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.433 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:11.455 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:11.461 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.469 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:11.482 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:58:11.492 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:11.500 DEBUG:teuthology.orchestra.run.smithi099:osd.9> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.9.service 2024-05-14T16:58:11.510 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-05-14T16:58:11.517 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:58:11.770 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:58:12.194 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:11 smithi193 ceph-mon[35962]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:12.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:11 smithi193 ceph-mon[35962]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:58:12.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:11 smithi193 ceph-mon[35962]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T16:58:12.238 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:11 smithi193 ceph-mon[35962]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:58:12.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:12.284 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:11 smithi099 ceph-mon[31851]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:12.294 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:11 smithi099 ceph-mon[31851]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:58:12.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:11 smithi099 ceph-mon[31851]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T16:58:12.319 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:11 smithi099 ceph-mon[31851]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:58:12.336 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:12.351 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:11 smithi150 ceph-mon[36204]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:12.358 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:11 smithi150 ceph-mon[36204]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:58:12.365 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:11 smithi150 ceph-mon[36204]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T16:58:12.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:11 smithi150 ceph-mon[36204]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-14T16:58:12.379 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:13.015 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 16:58:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T16:58:12.716+0000 7f95c0c88700 -1 osd.9 0 waiting for initial osdmap 2024-05-14T16:58:13.022 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 16:58:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T16:58:12.728+0000 7f95bca7b700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:58:13.555 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:58:13.563 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:58:13.570 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-14T16:58:13.577 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-14T16:58:13.608 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129145 s, 812 MB/s 2024-05-14T16:58:13.617 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-14T16:58:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:13 smithi099 ceph-mon[31851]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:13.781 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:13 smithi099 ceph-mon[31851]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:58:13.788 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:13 smithi099 ceph-mon[31851]: osdmap e60: 10 total, 9 up, 10 in 2024-05-14T16:58:13.819 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:13.826 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:13 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:13.835 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:13 smithi193 ceph-mon[35962]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:13.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:13 smithi193 ceph-mon[35962]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:58:13.849 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:13 smithi193 ceph-mon[35962]: osdmap e60: 10 total, 9 up, 10 in 2024-05-14T16:58:13.855 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:13.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:13 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:14.168 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:13 smithi150 ceph-mon[36204]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:58:14.176 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:13 smithi150 ceph-mon[36204]: from='osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-14T16:58:14.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:13 smithi150 ceph-mon[36204]: osdmap e60: 10 total, 9 up, 10 in 2024-05-14T16:58:14.190 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:14.197 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:13 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:14.242 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-05-14T16:58:14.489 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:58:14.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:58:14.737 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:58:14.745 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] boot 2024-05-14T16:58:14.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T16:58:14.792 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:14.799 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi099 2024-05-14T16:58:14.806 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:14.812 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:14.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:14.825 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:14.831 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:14.837 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:14.844 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi099 to 4272M 2024-05-14T16:58:14.850 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:14.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:14.902 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:14.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:14 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:58:15.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:58:15.031 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] boot 2024-05-14T16:58:15.048 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T16:58:15.055 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:15.061 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi099 2024-05-14T16:58:15.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.082 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.088 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.095 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.102 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi099 to 4272M 2024-05-14T16:58:15.117 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.132 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:15.150 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:15.157 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:14 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:58:15.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:58:15.188 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: osd.9 [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] boot 2024-05-14T16:58:15.203 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T16:58:15.210 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:58:15.216 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi099 2024-05-14T16:58:15.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.243 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.249 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.261 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:15.266 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi099 to 4272M 2024-05-14T16:58:15.271 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.282 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:15.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:15.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:14 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:15.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 -- 172.21.15.150:0/2458037333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 msgr2=0x7f960010a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:15.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 --2- 172.21.15.150:0/2458037333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f960010a900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b0d0 tx=0x7f95f800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 -- 172.21.15.150:0/2458037333 shutdown_connections 2024-05-14T16:58:16.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 --2- 172.21.15.150:0/2458037333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f960010a900 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 --2- 172.21.15.150:0/2458037333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96001024b0 0x7f96001028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.043 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 --2- 172.21.15.150:0/2458037333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96001036b0 0x7f9600103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 -- 172.21.15.150:0/2458037333 >> 172.21.15.150:0/2458037333 conn(0x7f96000fda70 msgr2=0x7f96000ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:16.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 -- 172.21.15.150:0/2458037333 shutdown_connections 2024-05-14T16:58:16.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.914+0000 7f960513d700 1 -- 172.21.15.150:0/2458037333 wait complete. 2024-05-14T16:58:16.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.915+0000 7f960513d700 1 Processor -- start 2024-05-14T16:58:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.915+0000 7f960513d700 1 -- start start 2024-05-14T16:58:16.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96001024b0 0x7f96001995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96001036b0 0x7f9600199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f96001a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f960010df30 con 0x7f96001036b0 2024-05-14T16:58:16.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f960010ddb0 con 0x7f9600104050 2024-05-14T16:58:16.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f960010e0b0 con 0x7f96001024b0 2024-05-14T16:58:16.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f96001a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f96001a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:52352/0 (socket says 172.21.15.150:52352) 2024-05-14T16:58:16.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 -- 172.21.15.150:0/3130332238 learned_addr learned my addr 172.21.15.150:0/3130332238 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-14T16:58:16.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95fe59c700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96001036b0 0x7f9600199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95fed9d700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96001024b0 0x7f96001995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 -- 172.21.15.150:0/3130332238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96001024b0 msgr2=0x7f96001995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:16.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96001024b0 0x7f96001995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 -- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96001036b0 msgr2=0x7f9600199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:16.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96001036b0 0x7f9600199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 -- 172.21.15.150:0/3130332238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96001a1240 con 0x7f9600104050 2024-05-14T16:58:16.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95ff59e700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f96001a0ba0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009040 tx=0x7f95f8004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:16.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f801f030 con 0x7f9600104050 2024-05-14T16:58:16.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.916+0000 7f960513d700 1 -- 172.21.15.150:0/3130332238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96001a14d0 con 0x7f9600104050 2024-05-14T16:58:16.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.917+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f8003a00 con 0x7f9600104050 2024-05-14T16:58:16.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.917+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8004670 con 0x7f9600104050 2024-05-14T16:58:16.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.917+0000 7f960513d700 1 -- 172.21.15.150:0/3130332238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a1980 con 0x7f9600104050 2024-05-14T16:58:16.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.917+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960006a230 con 0x7f9600104050 2024-05-14T16:58:16.331 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:15 smithi193 ceph-mon[35962]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:16.339 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:15 smithi193 ceph-mon[35962]: osdmap e62: 10 total, 10 up, 10 in 2024-05-14T16:58:16.345 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:15 smithi099 ceph-mon[31851]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:16.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:15 smithi099 ceph-mon[31851]: osdmap e62: 10 total, 10 up, 10 in 2024-05-14T16:58:16.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.919+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f95f8003bb0 con 0x7f9600104050 2024-05-14T16:58:16.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.919+0000 7f95e7fff700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f95e80b2880 0x7f95e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.919+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f95f800f020 con 0x7f9600104050 2024-05-14T16:58:16.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.920+0000 7f95fed9d700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f95e80b2880 0x7f95e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.920+0000 7f95fed9d700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f95e80b2880 0x7f95e80b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9600103510 tx=0x7f95f0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:16.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:15.923+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95f8029020 con 0x7f9600104050 2024-05-14T16:58:16.429 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:15 smithi150 ceph-mon[36204]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:16.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:15 smithi150 ceph-mon[36204]: osdmap e62: 10 total, 10 up, 10 in 2024-05-14T16:58:16.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:16.132+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f96000639f0 con 0x7f95e80b2880 2024-05-14T16:58:17.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:16 smithi099 ceph-mon[31851]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:58:17.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:16 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:58:17.082 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:16 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:58:17.094 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:16 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:17.102 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:16 smithi150 ceph-mon[36204]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:58:17.108 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:16 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:58:17.115 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:16 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:58:17.151 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:16 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:17.157 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:16 smithi193 ceph-mon[35962]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:58:17.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:16 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:58:17.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:16 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:58:17.176 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:16 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:18.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:17 smithi099 ceph-mon[31851]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:18.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:17 smithi193 ceph-mon[35962]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:18.085 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:17 smithi150 ceph-mon[36204]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:20.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:20.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/2872967557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]: dispatch 2024-05-14T16:58:20.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]: dispatch 2024-05-14T16:58:20.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]': finished 2024-05-14T16:58:20.112 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:58:20.119 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:20.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.150:0/2370929750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:58:20.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:20.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/2872967557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]: dispatch 2024-05-14T16:58:20.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]: dispatch 2024-05-14T16:58:20.189 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]': finished 2024-05-14T16:58:20.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:58:20.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:20.207 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.150:0/2370929750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:58:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:20.277 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/2872967557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]: dispatch 2024-05-14T16:58:20.283 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]: dispatch 2024-05-14T16:58:20.287 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64fe2c42-8ae1-4465-a821-ff6118d2a262"}]': finished 2024-05-14T16:58:20.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:58:20.326 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:20.332 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.150:0/2370929750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:58:22.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:21 smithi193 ceph-mon[35962]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:22.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:21 smithi150 ceph-mon[36204]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:22.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:21 smithi099 ceph-mon[31851]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:23.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:23 smithi150 ceph-mon[36204]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:24.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:23 smithi193 ceph-mon[35962]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:23 smithi099 ceph-mon[31851]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:26.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:25 smithi193 ceph-mon[35962]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:26.097 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:25 smithi150 ceph-mon[36204]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:26.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:25 smithi099 ceph-mon[31851]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:27.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:58:27.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:26 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:27.063 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:58:27.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:26 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:58:27.272 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:26 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:28.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:27 smithi099 ceph-mon[31851]: Deploying daemon osd.10 on smithi150 2024-05-14T16:58:28.023 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:27 smithi099 ceph-mon[31851]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:28.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:27 smithi193 ceph-mon[35962]: Deploying daemon osd.10 on smithi150 2024-05-14T16:58:28.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:27 smithi193 ceph-mon[35962]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:28.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:27 smithi150 ceph-mon[36204]: Deploying daemon osd.10 on smithi150 2024-05-14T16:58:28.146 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:27 smithi150 ceph-mon[36204]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:30.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:29 smithi150 ceph-mon[36204]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:30.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:29 smithi099 ceph-mon[31851]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:30.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:29 smithi193 ceph-mon[35962]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:32.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:31 smithi150 ceph-mon[36204]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:31 smithi099 ceph-mon[31851]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:32.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:31 smithi193 ceph-mon[35962]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:33.475 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:33.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:33 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:33.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:33.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:33.527 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:33 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:33.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:33.548 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:33 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:34.350 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:34 smithi150 ceph-mon[36204]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:34.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:34 smithi099 ceph-mon[31851]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:34.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:34 smithi193 ceph-mon[35962]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:35.699 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-05-14T16:58:35.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.699+0000 7f95e7fff700 1 -- 172.21.15.150:0/3130332238 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f96000639f0 con 0x7f95e80b2880 2024-05-14T16:58:35.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f95e80b2880 msgr2=0x7f95e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:35.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f95e80b2880 0x7f95e80b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9600103510 tx=0x7f95f0007400 comp rx=0 tx=0).stop 2024-05-14T16:58:35.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 msgr2=0x7f96001a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:35.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f96001a0ba0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009040 tx=0x7f95f8004d50 comp rx=0 tx=0).stop 2024-05-14T16:58:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 shutdown_connections 2024-05-14T16:58:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f95e80b2880 0x7f95e80b4d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:35.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9600104050 0x7f96001a0ba0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:35.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96001024b0 0x7f96001995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:35.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 --2- 172.21.15.150:0/3130332238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96001036b0 0x7f9600199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:35.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 >> 172.21.15.150:0/3130332238 conn(0x7f96000fda70 msgr2=0x7f96000ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:35.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 shutdown_connections 2024-05-14T16:58:35.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-14T16:58:35.702+0000 7f95e5ffb700 1 -- 172.21.15.150:0/3130332238 wait complete. 2024-05-14T16:58:35.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:35.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.004 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.010 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:36.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:36.041 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.046 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.053 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.058 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:36.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:36.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.293 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.300 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:36.307 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:36.329 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.354 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.367 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:36.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:36.405 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.444 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.451 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:36.458 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:36.467 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.10.service 2024-05-14T16:58:36.475 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-05-14T16:58:36.480 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:58:36.735 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:58:37.150 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:36 smithi150 ceph-mon[36204]: from='osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:58:37.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:36 smithi150 ceph-mon[36204]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:58:37.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:36 smithi193 ceph-mon[35962]: from='osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:58:37.178 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:36 smithi193 ceph-mon[35962]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:58:37.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:36 smithi099 ceph-mon[31851]: from='osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:58:37.271 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:36 smithi099 ceph-mon[31851]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:58:38.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:37 smithi150 ceph-mon[36204]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:38.210 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:37 smithi150 ceph-mon[36204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:58:38.217 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:37 smithi150 ceph-mon[36204]: from='osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:58:38.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:37 smithi150 ceph-mon[36204]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T16:58:38.231 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:37 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:38.239 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:37 smithi150 ceph-mon[36204]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:58:38.268 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 16:58:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T16:58:37.904+0000 7fabe1caa700 -1 osd.10 0 waiting for initial osdmap 2024-05-14T16:58:38.274 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 16:58:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T16:58:37.915+0000 7fabdda9d700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:58:38.281 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:37 smithi099 ceph-mon[31851]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:38.287 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:37 smithi099 ceph-mon[31851]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:58:38.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:37 smithi099 ceph-mon[31851]: from='osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:58:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:37 smithi099 ceph-mon[31851]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T16:58:38.366 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:37 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:38.373 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:37 smithi099 ceph-mon[31851]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:58:38.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:37 smithi193 ceph-mon[35962]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:38.435 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:37 smithi193 ceph-mon[35962]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:58:38.442 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:37 smithi193 ceph-mon[35962]: from='osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:58:38.449 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:37 smithi193 ceph-mon[35962]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T16:58:38.475 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:37 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:38.483 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:37 smithi193 ceph-mon[35962]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-14T16:58:38.508 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:58:38.516 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:58:38.523 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T16:58:38.533 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T16:58:38.539 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114667 s, 914 MB/s 2024-05-14T16:58:38.546 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T16:58:39.040 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-05-14T16:58:39.141 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:38 smithi150 ceph-mon[36204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:58:39.149 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:38 smithi150 ceph-mon[36204]: osdmap e65: 11 total, 10 up, 11 in 2024-05-14T16:58:39.156 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:38 smithi099 ceph-mon[31851]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:58:39.298 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:38 smithi099 ceph-mon[31851]: osdmap e65: 11 total, 10 up, 11 in 2024-05-14T16:58:39.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:39.311 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:39.334 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:38 smithi193 ceph-mon[35962]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-14T16:58:39.341 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:38 smithi193 ceph-mon[35962]: osdmap e65: 11 total, 10 up, 11 in 2024-05-14T16:58:39.348 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:39.355 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:39.361 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:58:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:58:40.177 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:58:40.184 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:40.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] boot 2024-05-14T16:58:40.255 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:58:40.260 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:40.266 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi150 2024-05-14T16:58:40.273 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.279 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.339 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.346 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.367 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.375 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-14T16:58:40.381 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.388 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:40.395 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:40.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.436 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:39 smithi193 ceph-mon[35962]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T16:58:40.444 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:58:40.450 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:58:40.471 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:40.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] boot 2024-05-14T16:58:40.497 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:58:40.504 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:40.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi150 2024-05-14T16:58:40.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.535 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.543 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.559 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.571 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.577 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-14T16:58:40.584 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.600 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:40.607 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:40.614 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.621 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:39 smithi099 ceph-mon[31851]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T16:58:40.636 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:58:40.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:58:40.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:58:40.679 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: osd.10 [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] boot 2024-05-14T16:58:40.686 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:58:40.701 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:58:40.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi150 2024-05-14T16:58:40.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.735 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.743 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.756 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:58:40.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-14T16:58:40.793 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.800 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:40.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:40.845 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:40.854 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:39 smithi150 ceph-mon[36204]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T16:58:41.032 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.018+0000 7feb54ba6700 1 -- 172.21.15.193:0/205148676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50104790 msgr2=0x7feb50104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:41.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.018+0000 7feb54ba6700 1 --2- 172.21.15.193:0/205148676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50104790 0x7feb50104bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feb44009990 tx=0x7feb44009ca0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 -- 172.21.15.193:0/205148676 shutdown_connections 2024-05-14T16:58:41.058 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 --2- 172.21.15.193:0/205148676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50105130 0x7feb5010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.064 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 --2- 172.21.15.193:0/205148676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb50103590 0x7feb501039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 --2- 172.21.15.193:0/205148676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50104790 0x7feb50104bf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 -- 172.21.15.193:0/205148676 >> 172.21.15.193:0/205148676 conn(0x7feb500feb30 msgr2=0x7feb50100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:41.099 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 -- 172.21.15.193:0/205148676 shutdown_connections 2024-05-14T16:58:41.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 -- 172.21.15.193:0/205148676 wait complete. 2024-05-14T16:58:41.114 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 Processor -- start 2024-05-14T16:58:41.120 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.019+0000 7feb54ba6700 1 -- start start 2024-05-14T16:58:41.131 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb50103590 0x7feb50199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.139 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50104790 0x7feb50199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.146 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 0x7feb501a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb5010f080 con 0x7feb50103590 2024-05-14T16:58:41.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb5010ef00 con 0x7feb50104790 2024-05-14T16:58:41.192 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb5010f200 con 0x7feb50105130 2024-05-14T16:58:41.199 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 0x7feb501a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 0x7feb501a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:34878/0 (socket says 172.21.15.193:34878) 2024-05-14T16:58:41.213 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 -- 172.21.15.193:0/3589001714 learned_addr learned my addr 172.21.15.193:0/3589001714 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:58:41.233 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4dd9b700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50104790 0x7feb50199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4e59c700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb50103590 0x7feb50199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.250 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 -- 172.21.15.193:0/3589001714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50104790 msgr2=0x7feb50199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:41.260 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50104790 0x7feb50199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.267 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 -- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb50103590 msgr2=0x7feb50199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:41.273 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb50103590 0x7feb50199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 -- 172.21.15.193:0/3589001714 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb501a12f0 con 0x7feb50105130 2024-05-14T16:58:41.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4dd9b700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50104790 0x7feb50199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:41.310 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb4ed9d700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 0x7feb501a0c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b9a0 tx=0x7feb4800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:41.325 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48019070 con 0x7feb50105130 2024-05-14T16:58:41.332 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb501a15e0 con 0x7feb50105130 2024-05-14T16:58:41.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb48014440 con 0x7feb50105130 2024-05-14T16:58:41.373 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4800ce60 con 0x7feb50105130 2024-05-14T16:58:41.381 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.020+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb501a1b20 con 0x7feb50105130 2024-05-14T16:58:41.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.021+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5006a230 con 0x7feb50105130 2024-05-14T16:58:41.396 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.023+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7feb4801c070 con 0x7feb50105130 2024-05-14T16:58:41.403 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.023+0000 7feb3f7fe700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feb380b2880 0x7feb380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.436 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.023+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7feb480e1490 con 0x7feb50105130 2024-05-14T16:58:41.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.024+0000 7feb4e59c700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feb380b2880 0x7feb380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.024+0000 7feb4e59c700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feb380b2880 0x7feb380b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb40005fd0 tx=0x7feb40005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:41.469 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.027+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb480a9890 con 0x7feb50105130 2024-05-14T16:58:41.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:58:41.239+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feb50063980 con 0x7feb380b2880 2024-05-14T16:58:41.956 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:41 smithi193 ceph-mon[35962]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:41.963 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:41 smithi193 ceph-mon[35962]: osdmap e68: 11 total, 11 up, 11 in 2024-05-14T16:58:41.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:41 smithi193 ceph-mon[35962]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:58:42.010 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:41 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:58:42.018 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:41 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:58:42.025 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:41 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:42.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:41 smithi099 ceph-mon[31851]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:42.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:41 smithi099 ceph-mon[31851]: osdmap e68: 11 total, 11 up, 11 in 2024-05-14T16:58:42.070 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:41 smithi099 ceph-mon[31851]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:58:42.079 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:58:42.087 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:58:42.095 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:42.158 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:41 smithi150 ceph-mon[36204]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:42.174 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:41 smithi150 ceph-mon[36204]: osdmap e68: 11 total, 11 up, 11 in 2024-05-14T16:58:42.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:41 smithi150 ceph-mon[36204]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:58:42.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:41 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:58:42.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:41 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:58:42.207 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:41 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:44.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:43 smithi099 ceph-mon[31851]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:44.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:43 smithi193 ceph-mon[35962]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:44.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:43 smithi150 ceph-mon[36204]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:45.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:44 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/1891518696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]: dispatch 2024-05-14T16:58:45.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:44 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]: dispatch 2024-05-14T16:58:45.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:44 smithi099 ceph-mon[31851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]': finished 2024-05-14T16:58:45.295 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:44 smithi099 ceph-mon[31851]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:58:45.322 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:44 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:58:45.355 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:44 smithi099 ceph-mon[31851]: from='client.? 172.21.15.193:0/2015321939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:58:45.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:44 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/1891518696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]: dispatch 2024-05-14T16:58:45.377 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:44 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]: dispatch 2024-05-14T16:58:45.473 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:44 smithi150 ceph-mon[36204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]': finished 2024-05-14T16:58:45.481 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:44 smithi150 ceph-mon[36204]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:58:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:44 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:58:45.536 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:44 smithi150 ceph-mon[36204]: from='client.? 172.21.15.193:0/2015321939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:58:45.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:44 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/1891518696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]: dispatch 2024-05-14T16:58:45.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:44 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]: dispatch 2024-05-14T16:58:45.640 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:44 smithi193 ceph-mon[35962]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc26027a-0163-4ecd-88da-2029accfe161"}]': finished 2024-05-14T16:58:45.646 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:44 smithi193 ceph-mon[35962]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:58:45.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:44 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:58:45.723 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:44 smithi193 ceph-mon[35962]: from='client.? 172.21.15.193:0/2015321939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:58:46.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:45 smithi150 ceph-mon[36204]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:46.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:45 smithi099 ceph-mon[31851]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:46.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:45 smithi193 ceph-mon[35962]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:58:48.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:47 smithi150 ceph-mon[36204]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-14T16:58:48.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:47 smithi099 ceph-mon[31851]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-14T16:58:48.280 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:47 smithi193 ceph-mon[35962]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-14T16:58:50.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:49 smithi150 ceph-mon[36204]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:58:50.227 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:49 smithi193 ceph-mon[35962]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:58:50.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:49 smithi099 ceph-mon[31851]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:58:52.183 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:51 smithi193 ceph-mon[35962]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:58:52.212 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:51 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:58:52.218 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:51 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:52.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:51 smithi150 ceph-mon[36204]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:58:52.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:51 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:58:52.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:51 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:52.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:51 smithi099 ceph-mon[31851]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T16:58:52.271 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:51 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:58:52.278 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:51 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:53.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:52 smithi150 ceph-mon[36204]: Deploying daemon osd.11 on smithi193 2024-05-14T16:58:53.232 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:52 smithi193 ceph-mon[35962]: Deploying daemon osd.11 on smithi193 2024-05-14T16:58:53.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:52 smithi099 ceph-mon[31851]: Deploying daemon osd.11 on smithi193 2024-05-14T16:58:54.151 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:53 smithi193 ceph-mon[35962]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:58:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:53 smithi150 ceph-mon[36204]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:58:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:53 smithi099 ceph-mon[31851]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:58:56.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:55 smithi193 ceph-mon[35962]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-14T16:58:56.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:55 smithi150 ceph-mon[36204]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-14T16:58:56.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:55 smithi099 ceph-mon[31851]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-14T16:58:58.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:57 smithi150 ceph-mon[36204]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:58:58.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:58.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:58.287 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:57 smithi193 ceph-mon[35962]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:58:58.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:58.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:58.331 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:58.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:57 smithi099 ceph-mon[31851]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:58:58.345 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:58.352 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:58:58.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:00.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:59 smithi150 ceph-mon[36204]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:00.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.221 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.226 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:00.231 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:00.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:58:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:59 smithi099 ceph-mon[31851]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:00.330 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.342 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.347 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:00.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:00.367 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:58:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.374 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:59 smithi193 ceph-mon[35962]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:00.380 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.387 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.393 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:00.399 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:00.404 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:58:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:00.606 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.606+0000 7feb3f7fe700 1 -- 172.21.15.193:0/3589001714 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feb50063980 con 0x7feb380b2880 2024-05-14T16:59:00.621 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-05-14T16:59:00.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feb380b2880 msgr2=0x7feb380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:00.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feb380b2880 0x7feb380b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb40005fd0 tx=0x7feb40005e50 comp rx=0 tx=0).stop 2024-05-14T16:59:00.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 msgr2=0x7feb501a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:00.675 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 0x7feb501a0c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b9a0 tx=0x7feb4800bd60 comp rx=0 tx=0).stop 2024-05-14T16:59:00.682 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 shutdown_connections 2024-05-14T16:59:00.689 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feb380b2880 0x7feb380b4d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb50105130 0x7feb501a0c50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.703 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb50103590 0x7feb50199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.735 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 --2- 172.21.15.193:0/3589001714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb50104790 0x7feb50199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:00.742 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 >> 172.21.15.193:0/3589001714 conn(0x7feb500feb30 msgr2=0x7feb50107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:00.749 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 shutdown_connections 2024-05-14T16:59:00.755 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:00.609+0000 7feb54ba6700 1 -- 172.21.15.193:0/3589001714 wait complete. 2024-05-14T16:59:01.187 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.11.service 2024-05-14T16:59:01.198 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-14T16:59:01.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-14T16:59:01.461 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:01.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:01 smithi193 ceph-mon[35962]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:01.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:01.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:01.868 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:01 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:01.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:01 smithi193 ceph-mon[35962]: from='osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:59:01.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:01 smithi193 ceph-mon[35962]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:59:01.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:01 smithi099 ceph-mon[31851]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:01.901 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:01.908 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:01.915 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:01 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:01.922 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:01 smithi099 ceph-mon[31851]: from='osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:59:01.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:01 smithi099 ceph-mon[31851]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:59:01.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:01 smithi150 ceph-mon[36204]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:01.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:01.966 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:01.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:01 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:01.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:01 smithi150 ceph-mon[36204]: from='osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:59:02.006 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:01 smithi150 ceph-mon[36204]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:59:02.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:02 smithi193 ceph-mon[35962]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:59:02.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:02 smithi193 ceph-mon[35962]: from='osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:59:02.867 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:02 smithi193 ceph-mon[35962]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T16:59:02.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:02.883 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:02 smithi193 ceph-mon[35962]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:59:02.890 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 16:59:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T16:59:02.605+0000 7fb14f6c3700 -1 osd.11 0 waiting for initial osdmap 2024-05-14T16:59:02.900 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 16:59:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T16:59:02.622+0000 7fb14b4b6700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:59:02.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:02 smithi099 ceph-mon[31851]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:59:02.914 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:02 smithi099 ceph-mon[31851]: from='osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:59:02.921 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:02 smithi099 ceph-mon[31851]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T16:59:02.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:02.936 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:02 smithi099 ceph-mon[31851]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:59:02.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:02 smithi150 ceph-mon[36204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:59:02.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:02 smithi150 ceph-mon[36204]: from='osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:59:02.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:02 smithi150 ceph-mon[36204]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T16:59:02.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:03.005 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:02 smithi150 ceph-mon[36204]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T16:59:03.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.060+0000 7f83666df700 1 -- 172.21.15.99:0/1320188977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 msgr2=0x7f836010b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:03.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 --2- 172.21.15.99:0/1320188977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f836010b9e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f835800b0d0 tx=0x7f835800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 -- 172.21.15.99:0/1320188977 shutdown_connections 2024-05-14T16:59:03.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 --2- 172.21.15.99:0/1320188977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f836010b9e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 --2- 172.21.15.99:0/1320188977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8360103590 0x7f83601039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 --2- 172.21.15.99:0/1320188977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8360104790 0x7f8360104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 -- 172.21.15.99:0/1320188977 >> 172.21.15.99:0/1320188977 conn(0x7f83600feb30 msgr2=0x7f8360100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:03.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 -- 172.21.15.99:0/1320188977 shutdown_connections 2024-05-14T16:59:03.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 -- 172.21.15.99:0/1320188977 wait complete. 2024-05-14T16:59:03.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 Processor -- start 2024-05-14T16:59:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.061+0000 7f83666df700 1 -- start start 2024-05-14T16:59:03.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8360103590 0x7f83600772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:03.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8360104790 0x7f8360075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:03.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f8360075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:03.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f836010f030 con 0x7f8360105130 2024-05-14T16:59:03.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f836010eeb0 con 0x7f8360103590 2024-05-14T16:59:03.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f836010f1b0 con 0x7f8360104790 2024-05-14T16:59:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f8360075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:03.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f8360075e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44514/0 (socket says 172.21.15.99:44514) 2024-05-14T16:59:03.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 -- 172.21.15.99:0/3368995945 learned_addr learned my addr 172.21.15.99:0/3368995945 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:03.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f835f7fe700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8360104790 0x7f8360075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:03.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f835ffff700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8360103590 0x7f83600772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 -- 172.21.15.99:0/3368995945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8360104790 msgr2=0x7f8360075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8360104790 0x7f8360075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 -- 172.21.15.99:0/3368995945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8360103590 msgr2=0x7f83600772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:03.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8360103590 0x7f83600772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 -- 172.21.15.99:0/3368995945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8360076720 con 0x7f8360105130 2024-05-14T16:59:03.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f8364c7c700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f8360075e70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8358009cd0 tx=0x7f8358004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:03.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835801f030 con 0x7f8360105130 2024-05-14T16:59:03.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8358005040 con 0x7f8360105130 2024-05-14T16:59:03.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83601a3750 con 0x7f8360105130 2024-05-14T16:59:03.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835801b670 con 0x7f8360105130 2024-05-14T16:59:03.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.062+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83601a3bb0 con 0x7f8360105130 2024-05-14T16:59:03.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.063+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836006a230 con 0x7f8360105130 2024-05-14T16:59:03.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.065+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8358022030 con 0x7f8360105130 2024-05-14T16:59:03.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.065+0000 7f835d7fa700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f83480b2880 0x7f83480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:03.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.066+0000 7f835ffff700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f83480b2880 0x7f83480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:03.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.066+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f835800f020 con 0x7f8360105130 2024-05-14T16:59:03.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.066+0000 7f835ffff700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f83480b2880 0x7f83480b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83601045f0 tx=0x7f8350009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.069+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8358029020 con 0x7f8360105130 2024-05-14T16:59:03.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.270+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f836002e600 con 0x7f8360105130 2024-05-14T16:59:03.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.270+0000 7f835d7fa700 1 -- 172.21.15.99:0/3368995945 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f836002e600 con 0x7f8360105130 2024-05-14T16:59:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:03.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f83480b2880 msgr2=0x7f83480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:03.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f83480b2880 0x7f83480b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83601045f0 tx=0x7f8350009260 comp rx=0 tx=0).stop 2024-05-14T16:59:03.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 msgr2=0x7f8360075e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:03.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f8360075e70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8358009cd0 tx=0x7f8358004d70 comp rx=0 tx=0).stop 2024-05-14T16:59:03.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 shutdown_connections 2024-05-14T16:59:03.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f83480b2880 0x7f83480b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8360105130 0x7f8360075e70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8360103590 0x7f83600772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 --2- 172.21.15.99:0/3368995945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8360104790 0x7f8360075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 >> 172.21.15.99:0/3368995945 conn(0x7f83600feb30 msgr2=0x7f8360107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:03.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 shutdown_connections 2024-05-14T16:59:03.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:03.274+0000 7f83666df700 1 -- 172.21.15.99:0/3368995945 wait complete. 2024-05-14T16:59:03.877 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:03 smithi193 ceph-mon[35962]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:03.885 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:03 smithi193 ceph-mon[35962]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:59:03.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:03 smithi193 ceph-mon[35962]: osdmap e71: 12 total, 11 up, 12 in 2024-05-14T16:59:03.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:03.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:03 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3368995945' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:59:03.926 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:03 smithi099 ceph-mon[31851]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:03.932 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:03 smithi099 ceph-mon[31851]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:59:03.939 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:03 smithi099 ceph-mon[31851]: osdmap e71: 12 total, 11 up, 12 in 2024-05-14T16:59:03.945 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:03.959 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:03.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:03 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3368995945' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:59:03.985 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:03 smithi150 ceph-mon[36204]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:03.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:03 smithi150 ceph-mon[36204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T16:59:03.998 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:03 smithi150 ceph-mon[36204]: osdmap e71: 12 total, 11 up, 12 in 2024-05-14T16:59:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:04.036 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:04.043 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:03 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3368995945' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:59:04.050 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715705918,"num_in_osds":12,"osd_in_since":1715705923,"num_remapped_pgs":0} 2024-05-14T16:59:04.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: purged_snaps scrub starts 2024-05-14T16:59:04.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: purged_snaps scrub ok 2024-05-14T16:59:05.016 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:05.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] boot 2024-05-14T16:59:05.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:59:05.035 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:05.041 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: Detected new or changed devices on smithi193 2024-05-14T16:59:05.048 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.061 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.075 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.103 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.152 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi193 to 4272M 2024-05-14T16:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.168 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:05.174 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:05.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.188 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-14T16:59:05.215 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: purged_snaps scrub starts 2024-05-14T16:59:05.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: purged_snaps scrub ok 2024-05-14T16:59:05.229 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:05.235 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] boot 2024-05-14T16:59:05.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:59:05.247 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:05.252 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: Detected new or changed devices on smithi193 2024-05-14T16:59:05.297 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.331 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.338 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.348 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:05.354 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi193 to 4272M 2024-05-14T16:59:05.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:05.407 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:06.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:06.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:06.949 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: purged_snaps scrub starts 2024-05-14T16:59:06.956 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: purged_snaps scrub ok 2024-05-14T16:59:06.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:06.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: osd.11 [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] boot 2024-05-14T16:59:06.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:59:06.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:59:06.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: Detected new or changed devices on smithi193 2024-05-14T16:59:06.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:07.051 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:07.057 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:07.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:07.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:07.080 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:59:07.087 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi193 to 4272M 2024-05-14T16:59:07.210 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:07.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:07.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:07.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:07.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:05 smithi150 ceph-mon[36204]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:07.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:05 smithi150 ceph-mon[36204]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:59:07.254 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:05 smithi193 ceph-mon[35962]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:07.260 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:05 smithi193 ceph-mon[35962]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:59:07.267 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:05 smithi099 ceph-mon[31851]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:59:07.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:05 smithi099 ceph-mon[31851]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:59:07.490 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:07.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:07 smithi099 ceph-mon[31851]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:08.011 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:07 smithi150 ceph-mon[36204]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:08.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:07 smithi193 ceph-mon[35962]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 -- 172.21.15.99:0/2943075280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01092f0 msgr2=0x7f34b010b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 --2- 172.21.15.99:0/2943075280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01092f0 0x7f34b010b700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b0a0 tx=0x7f34a800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 -- 172.21.15.99:0/2943075280 shutdown_connections 2024-05-14T16:59:09.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 --2- 172.21.15.99:0/2943075280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01092f0 0x7f34b010b700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 --2- 172.21.15.99:0/2943075280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b0104090 0x7f34b0106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 --2- 172.21.15.99:0/2943075280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b01069c0 0x7f34b0108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 -- 172.21.15.99:0/2943075280 >> 172.21.15.99:0/2943075280 conn(0x7f34b00fda40 msgr2=0x7f34b00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:09.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 -- 172.21.15.99:0/2943075280 shutdown_connections 2024-05-14T16:59:09.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.135+0000 7f34b61c4700 1 -- 172.21.15.99:0/2943075280 wait complete. 2024-05-14T16:59:09.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 Processor -- start 2024-05-14T16:59:09.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 -- start start 2024-05-14T16:59:09.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b0104090 0x7f34b01039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 0x7f34b0102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b01092f0 0x7f34b0102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010f080 con 0x7f34b01069c0 2024-05-14T16:59:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010ef00 con 0x7f34b01092f0 2024-05-14T16:59:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34b61c4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010f200 con 0x7f34b0104090 2024-05-14T16:59:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 0x7f34b0102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34a6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 0x7f34b0102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41038/0 (socket says 172.21.15.99:41038) 2024-05-14T16:59:09.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34a6ffd700 1 -- 172.21.15.99:0/1835148897 learned_addr learned my addr 172.21.15.99:0/1835148897 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.136+0000 7f34af7fe700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b0104090 0x7f34b01039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34affff700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b01092f0 0x7f34b0102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34a6ffd700 1 -- 172.21.15.99:0/1835148897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b0104090 msgr2=0x7f34b01039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34a6ffd700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b0104090 0x7f34b01039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34a6ffd700 1 -- 172.21.15.99:0/1835148897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b01092f0 msgr2=0x7f34b0102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34a6ffd700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b01092f0 0x7f34b0102580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34a6ffd700 1 -- 172.21.15.99:0/1835148897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b0102e30 con 0x7f34b01069c0 2024-05-14T16:59:09.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34af7fe700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b0104090 0x7f34b01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:09.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34affff700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b01092f0 0x7f34b0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:09.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34a6ffd700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 0x7f34b0102040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f349800c920 tx=0x7f349800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:09.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498007b00 con 0x7f34b01069c0 2024-05-14T16:59:09.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b01a38a0 con 0x7f34b01069c0 2024-05-14T16:59:09.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3498004ce0 con 0x7f34b01069c0 2024-05-14T16:59:09.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498005650 con 0x7f34b01069c0 2024-05-14T16:59:09.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.137+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a3e60 con 0x7f34b01069c0 2024-05-14T16:59:09.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.138+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b006a230 con 0x7f34b01069c0 2024-05-14T16:59:09.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.139+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3498007ca0 con 0x7f34b01069c0 2024-05-14T16:59:09.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.140+0000 7f34ad7fa700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f349c0b2880 0x7f349c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.140+0000 7f34af7fe700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f349c0b2880 0x7f349c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.140+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f34980e1ef0 con 0x7f34b01069c0 2024-05-14T16:59:09.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.140+0000 7f34af7fe700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f349c0b2880 0x7f349c0b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34a0007900 tx=0x7f34a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:09.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.144+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3498014030 con 0x7f34b01069c0 2024-05-14T16:59:09.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.345+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f34b002e590 con 0x7f34b01069c0 2024-05-14T16:59:09.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.345+0000 7f34ad7fa700 1 -- 172.21.15.99:0/1835148897 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f34980aa0b0 con 0x7f34b01069c0 2024-05-14T16:59:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:09.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.347+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f349c0b2880 msgr2=0x7f349c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.347+0000 7f34b61c4700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f349c0b2880 0x7f349c0b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f34a0007900 tx=0x7f34a0008040 comp rx=0 tx=0).stop 2024-05-14T16:59:09.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 msgr2=0x7f34b0102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 0x7f34b0102040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f349800c920 tx=0x7f349800cce0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 shutdown_connections 2024-05-14T16:59:09.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f349c0b2880 0x7f349c0b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b01092f0 0x7f34b0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b0104090 0x7f34b01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 --2- 172.21.15.99:0/1835148897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b01069c0 0x7f34b0102040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 >> 172.21.15.99:0/1835148897 conn(0x7f34b00fda40 msgr2=0x7f34b01049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 shutdown_connections 2024-05-14T16:59:09.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:09.348+0000 7f34b61c4700 1 -- 172.21.15.99:0/1835148897 wait complete. 2024-05-14T16:59:09.943 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:09 smithi099 ceph-mon[31851]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:09.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:09 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1835148897' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:59:09.957 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1715705943,"num_in_osds":12,"osd_in_since":1715705923,"num_remapped_pgs":0} 2024-05-14T16:59:09.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:59:09.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:09 smithi150 ceph-mon[36204]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:09.979 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:09 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1835148897' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:59:10.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:09 smithi193 ceph-mon[35962]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:10.059 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:09 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1835148897' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:59:10.243 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:11.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:11 smithi099 ceph-mon[31851]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:11.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 -- 172.21.15.99:0/603209113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 msgr2=0x7f2680110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:11.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 --2- 172.21.15.99:0/603209113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f2680110470 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f267800b0d0 tx=0x7f267800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 -- 172.21.15.99:0/603209113 shutdown_connections 2024-05-14T16:59:11.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 --2- 172.21.15.99:0/603209113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f2680110470 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 --2- 172.21.15.99:0/603209113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 0x7f2680103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 --2- 172.21.15.99:0/603209113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26801041a0 0x7f26801081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:11.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 -- 172.21.15.99:0/603209113 >> 172.21.15.99:0/603209113 conn(0x7f268007a6a0 msgr2=0x7f268007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:11.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 -- 172.21.15.99:0/603209113 shutdown_connections 2024-05-14T16:59:11.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.888+0000 7f2685615700 1 -- 172.21.15.99:0/603209113 wait complete. 2024-05-14T16:59:11.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 Processor -- start 2024-05-14T16:59:11.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 -- start start 2024-05-14T16:59:11.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 0x7f2680075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26801041a0 0x7f2680075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f26800762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26801136c0 con 0x7f26801088f0 2024-05-14T16:59:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2680113540 con 0x7f26801041a0 2024-05-14T16:59:12.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f2685615700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2680113840 con 0x7f26801037f0 2024-05-14T16:59:12.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f267f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f26800762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f267f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f26800762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41066/0 (socket says 172.21.15.99:41066) 2024-05-14T16:59:12.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f267f7fe700 1 -- 172.21.15.99:0/1863342928 learned_addr learned my addr 172.21.15.99:0/1863342928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:12.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.889+0000 7f26767fc700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26801041a0 0x7f2680075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267effd700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 0x7f2680075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267f7fe700 1 -- 172.21.15.99:0/1863342928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 msgr2=0x7f2680075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267f7fe700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 0x7f2680075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267f7fe700 1 -- 172.21.15.99:0/1863342928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26801041a0 msgr2=0x7f2680075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267f7fe700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26801041a0 0x7f2680075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267f7fe700 1 -- 172.21.15.99:0/1863342928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26801a80a0 con 0x7f26801088f0 2024-05-14T16:59:12.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267effd700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 0x7f2680075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:12.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267f7fe700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f26800762c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2678007a00 tx=0x7f2678007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:12.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267801f070 con 0x7f26801088f0 2024-05-14T16:59:12.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26801a8330 con 0x7f26801088f0 2024-05-14T16:59:12.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2678004c50 con 0x7f26801088f0 2024-05-14T16:59:12.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267801b690 con 0x7f26801088f0 2024-05-14T16:59:12.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.890+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801a8780 con 0x7f26801088f0 2024-05-14T16:59:12.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.891+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268006a230 con 0x7f26801088f0 2024-05-14T16:59:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.892+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2678022070 con 0x7f26801088f0 2024-05-14T16:59:12.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.893+0000 7f267cff9700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f266c0b2880 0x7f266c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:12.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.893+0000 7f267effd700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f266c0b2880 0x7f266c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:12.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.893+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f26780e2d50 con 0x7f26801088f0 2024-05-14T16:59:12.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.893+0000 7f267effd700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f266c0b2880 0x7f266c0b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2670009cb0 tx=0x7f2670009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:11.896+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26780aafa0 con 0x7f26801088f0 2024-05-14T16:59:12.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:11 smithi193 ceph-mon[35962]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:12.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:11 smithi150 ceph-mon[36204]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.101+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f268002e720 con 0x7f26801088f0 2024-05-14T16:59:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.102+0000 7f267cff9700 1 -- 172.21.15.99:0/1863342928 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18480 (secure 0 0 0) 0x7f26780aa8a0 con 0x7f26801088f0 2024-05-14T16:59:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:12.471 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":73,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T16:59:04.611085+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:59:12.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f266c0b2880 msgr2=0x7f266c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f266c0b2880 0x7f266c0b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2670009cb0 tx=0x7f2670009260 comp rx=0 tx=0).stop 2024-05-14T16:59:12.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 msgr2=0x7f26800762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:12.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f26800762c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2678007a00 tx=0x7f2678007a30 comp rx=0 tx=0).stop 2024-05-14T16:59:12.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 shutdown_connections 2024-05-14T16:59:12.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f266c0b2880 0x7f266c0b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801088f0 0x7f26800762c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26801037f0 0x7f2680075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 --2- 172.21.15.99:0/1863342928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26801041a0 0x7f2680075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:12.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 >> 172.21.15.99:0/1863342928 conn(0x7f268007a6a0 msgr2=0x7f2680101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:12.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 shutdown_connections 2024-05-14T16:59:12.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:12.105+0000 7f2685615700 1 -- 172.21.15.99:0/1863342928 wait complete. 2024-05-14T16:59:12.793 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T16:55:26.450956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T16:59:12.801 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-14T16:59:12.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:12 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1863342928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:59:12.967 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:12 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1863342928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:59:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:12 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1863342928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:59:13.044 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:13 smithi099 ceph-mon[31851]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:13.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:13 smithi150 ceph-mon[36204]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:14.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:13 smithi193 ceph-mon[35962]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:14.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 -- 172.21.15.99:0/2963328266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 msgr2=0x7fbea4109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 --2- 172.21.15.99:0/2963328266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4109a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe940099c0 tx=0x7fbe94009cd0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 -- 172.21.15.99:0/2963328266 shutdown_connections 2024-05-14T16:59:14.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 --2- 172.21.15.99:0/2963328266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea406c6d0 0x7fbea4110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 --2- 172.21.15.99:0/2963328266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4109a10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 --2- 172.21.15.99:0/2963328266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406bd10 0x7fbea406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 -- 172.21.15.99:0/2963328266 >> 172.21.15.99:0/2963328266 conn(0x7fbea40794e0 msgr2=0x7fbea40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 -- 172.21.15.99:0/2963328266 shutdown_connections 2024-05-14T16:59:14.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.533+0000 7fbeaa836700 1 -- 172.21.15.99:0/2963328266 wait complete. 2024-05-14T16:59:14.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 Processor -- start 2024-05-14T16:59:14.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 -- start start 2024-05-14T16:59:14.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea406bd10 0x7fbea419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c6d0 0x7fbea419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4113810 con 0x7fbea4109630 2024-05-14T16:59:14.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4113690 con 0x7fbea406c6d0 2024-05-14T16:59:14.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.534+0000 7fbeaa836700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4113990 con 0x7fbea406bd10 2024-05-14T16:59:14.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54954/0 (socket says 172.21.15.99:54954) 2024-05-14T16:59:14.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 -- 172.21.15.99:0/780551081 learned_addr learned my addr 172.21.15.99:0/780551081 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:14.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea3fff700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea406bd10 0x7fbea419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea37fe700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c6d0 0x7fbea419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 -- 172.21.15.99:0/780551081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea406bd10 msgr2=0x7fbea419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea406bd10 0x7fbea419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 -- 172.21.15.99:0/780551081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c6d0 msgr2=0x7fbea419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c6d0 0x7fbea419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 -- 172.21.15.99:0/780551081 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea419a230 con 0x7fbea4109630 2024-05-14T16:59:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea8dd3700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4199ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00b970 tx=0x7fbe9c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:14.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c019070 con 0x7fbea4109630 2024-05-14T16:59:14.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9c00cde0 con 0x7fbea4109630 2024-05-14T16:59:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea419a520 con 0x7fbea4109630 2024-05-14T16:59:14.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007370 con 0x7fbea4109630 2024-05-14T16:59:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.535+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea41a6790 con 0x7fbea4109630 2024-05-14T16:59:14.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.536+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea410bac0 con 0x7fbea4109630 2024-05-14T16:59:14.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.538+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbe9c01c070 con 0x7fbea4109630 2024-05-14T16:59:14.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.538+0000 7fbea17fa700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe8c0b2900 0x7fbe8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:14.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.539+0000 7fbea3fff700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe8c0b2900 0x7fbe8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.539+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fbe9c0e1940 con 0x7fbea4109630 2024-05-14T16:59:14.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.539+0000 7fbea3fff700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe8c0b2900 0x7fbe8c0b4dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbea41090f0 tx=0x7fbe9400b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:14.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.542+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe9c023070 con 0x7fbea4109630 2024-05-14T16:59:14.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.743+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbea419afb0 con 0x7fbea4109630 2024-05-14T16:59:14.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.744+0000 7fbea17fa700 1 -- 172.21.15.99:0/780551081 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fbe9c0a9b80 con 0x7fbea4109630 2024-05-14T16:59:14.910 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-05-14T16:59:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe8c0b2900 msgr2=0x7fbe8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe8c0b2900 0x7fbe8c0b4dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbea41090f0 tx=0x7fbe9400b870 comp rx=0 tx=0).stop 2024-05-14T16:59:14.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 msgr2=0x7fbea4199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4199ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00b970 tx=0x7fbe9c00bc80 comp rx=0 tx=0).stop 2024-05-14T16:59:14.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 shutdown_connections 2024-05-14T16:59:14.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe8c0b2900 0x7fbe8c0b4dc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:15.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea4109630 0x7fbea4199ad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:15.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea406bd10 0x7fbea419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:15.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 --2- 172.21.15.99:0/780551081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbea406c6d0 0x7fbea419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:15.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 >> 172.21.15.99:0/780551081 conn(0x7fbea40794e0 msgr2=0x7fbea41010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:15.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 shutdown_connections 2024-05-14T16:59:15.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:14.746+0000 7fbeaa836700 1 -- 172.21.15.99:0/780551081 wait complete. 2024-05-14T16:59:15.343 INFO:tasks.cephadm:Adding mds.a on smithi099 2024-05-14T16:59:15.349 INFO:tasks.cephadm:Adding mds.d on smithi099 2024-05-14T16:59:15.354 INFO:tasks.cephadm:Adding mds.g on smithi099 2024-05-14T16:59:15.358 INFO:tasks.cephadm:Adding mds.j on smithi099 2024-05-14T16:59:15.370 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-05-14T16:59:15.375 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-05-14T16:59:15.391 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-05-14T16:59:15.424 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-05-14T16:59:15.431 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-05-14T16:59:15.466 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-05-14T16:59:15.475 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-05-14T16:59:15.481 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-05-14T16:59:15.489 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-05-14T16:59:15.743 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:59:15.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:15 smithi193 ceph-mon[35962]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:15.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:15 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/780551081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:59:15.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:15 smithi150 ceph-mon[36204]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:15.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:15 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/780551081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:59:16.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:15 smithi099 ceph-mon[31851]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:16.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:15 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/780551081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:59:17.373 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 -- 172.21.15.193:0/3397510845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c105140 msgr2=0x7ff86c10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:17.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 --2- 172.21.15.193:0/3397510845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c105140 0x7ff86c10b9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff86400b0a0 tx=0x7ff86400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.386 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 -- 172.21.15.193:0/3397510845 shutdown_connections 2024-05-14T16:59:17.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 --2- 172.21.15.193:0/3397510845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c105140 0x7ff86c10b9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 --2- 172.21.15.193:0/3397510845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c1035a0 0x7ff86c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 --2- 172.21.15.193:0/3397510845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff86c1047a0 0x7ff86c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 -- 172.21.15.193:0/3397510845 >> 172.21.15.193:0/3397510845 conn(0x7ff86c0feb40 msgr2=0x7ff86c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:17.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 -- 172.21.15.193:0/3397510845 shutdown_connections 2024-05-14T16:59:17.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.373+0000 7ff8708ab700 1 -- 172.21.15.193:0/3397510845 wait complete. 2024-05-14T16:59:17.644 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 Processor -- start 2024-05-14T16:59:17.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 -- start start 2024-05-14T16:59:17.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff86c1035a0 0x7ff86c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:17.682 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 0x7ff86c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:17.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c105140 0x7ff86c1a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:17.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10f040 con 0x7ff86c1035a0 2024-05-14T16:59:17.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10eec0 con 0x7ff86c105140 2024-05-14T16:59:17.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.374+0000 7ff8708ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10f1c0 con 0x7ff86c1047a0 2024-05-14T16:59:17.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 0x7ff86c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:17.724 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 0x7ff86c19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53936/0 (socket says 172.21.15.193:53936) 2024-05-14T16:59:17.734 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 -- 172.21.15.193:0/1235507076 learned_addr learned my addr 172.21.15.193:0/1235507076 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T16:59:17.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff86ad9d700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c105140 0x7ff86c1a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:17.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff86a59c700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff86c1035a0 0x7ff86c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:17.794 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 -- 172.21.15.193:0/1235507076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c105140 msgr2=0x7ff86c1a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:17.812 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c105140 0x7ff86c1a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.819 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 -- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff86c1035a0 msgr2=0x7ff86c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:17.826 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff86c1035a0 0x7ff86c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.833 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 -- 172.21.15.193:0/1235507076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86c1a3520 con 0x7ff86c1047a0 2024-05-14T16:59:17.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff869d9b700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 0x7ff86c19be00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00eb60 tx=0x7ff85c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:17.846 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00cd40 con 0x7ff86c1047a0 2024-05-14T16:59:17.853 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85c0044d0 con 0x7ff86c1047a0 2024-05-14T16:59:17.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86c1a3810 con 0x7ff86c1047a0 2024-05-14T16:59:17.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c010690 con 0x7ff86c1047a0 2024-05-14T16:59:17.872 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.375+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c1a3da0 con 0x7ff86c1047a0 2024-05-14T16:59:17.878 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.376+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86c06a230 con 0x7ff86c1047a0 2024-05-14T16:59:17.885 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.377+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff85c0037a0 con 0x7ff86c1047a0 2024-05-14T16:59:17.892 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.378+0000 7ff85b7fe700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8540b2900 0x7ff8540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:17.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.378+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ff85c0e2000 con 0x7ff86c1047a0 2024-05-14T16:59:17.913 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.378+0000 7ff86a59c700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8540b2900 0x7ff8540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:17.928 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.379+0000 7ff86a59c700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8540b2900 0x7ff8540b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff860005950 tx=0x7ff8600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:17.935 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.382+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff85c0aa1c0 con 0x7ff86c1047a0 2024-05-14T16:59:17.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.597+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7ff86c076f90 con 0x7ff8540b2900 2024-05-14T16:59:17.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.604+0000 7ff85b7fe700 1 -- 172.21.15.193:0/1235507076 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff86c076f90 con 0x7ff8540b2900 2024-05-14T16:59:17.973 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-05-14T16:59:17.981 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8540b2900 msgr2=0x7ff8540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:17.986 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8540b2900 0x7ff8540b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff860005950 tx=0x7ff8600058e0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.992 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 msgr2=0x7ff86c19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:17.998 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 0x7ff86c19be00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00eb60 tx=0x7ff85c00ef20 comp rx=0 tx=0).stop 2024-05-14T16:59:18.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 shutdown_connections 2024-05-14T16:59:18.009 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8540b2900 0x7ff8540b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c105140 0x7ff86c1a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.024 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff86c1035a0 0x7ff86c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.029 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 --2- 172.21.15.193:0/1235507076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff86c1047a0 0x7ff86c19be00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.035 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 >> 172.21.15.193:0/1235507076 conn(0x7ff86c0feb40 msgr2=0x7ff86c075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 shutdown_connections 2024-05-14T16:59:18.055 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T16:59:17.607+0000 7ff8708ab700 1 -- 172.21.15.193:0/1235507076 wait complete. 2024-05-14T16:59:18.076 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:17 smithi193 ceph-mon[35962]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:18.082 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:18.104 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:17 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:18.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:17 smithi099 ceph-mon[31851]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:18.118 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:18.126 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:17 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:18.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:17 smithi150 ceph-mon[36204]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:18.137 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:18.143 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:17 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:18.216 DEBUG:teuthology.orchestra.run.smithi099:mds.a> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.a.service 2024-05-14T16:59:18.226 DEBUG:teuthology.orchestra.run.smithi099:mds.d> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.d.service 2024-05-14T16:59:18.276 DEBUG:teuthology.orchestra.run.smithi099:mds.g> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.g.service 2024-05-14T16:59:18.337 DEBUG:teuthology.orchestra.run.smithi099:mds.j> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.j.service 2024-05-14T16:59:18.389 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.b.service 2024-05-14T16:59:18.399 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.e.service 2024-05-14T16:59:18.449 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.h.service 2024-05-14T16:59:18.526 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.k.service 2024-05-14T16:59:18.576 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.c.service 2024-05-14T16:59:18.585 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service 2024-05-14T16:59:18.635 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.i.service 2024-05-14T16:59:18.684 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.l.service 2024-05-14T16:59:18.721 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:59:18.727 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T16:59:18.733 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:18.739 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:18.745 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:18.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:18.761 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:18.767 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:18 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:18.774 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-14T16:59:18.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:19.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:59:19.027 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T16:59:19.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:19.072 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:19.082 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:19.089 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:19.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:19.102 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:18 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:19.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:59:19.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T16:59:19.122 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:19.128 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:19.134 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:19.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:19.147 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:19.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:18 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:19.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:19 smithi099 ceph-mon[31851]: Deploying daemon mds.l on smithi193 2024-05-14T16:59:19.778 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:19 smithi099 ceph-mon[31851]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:19.896 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:19 smithi193 ceph-mon[35962]: Deploying daemon mds.l on smithi193 2024-05-14T16:59:19.904 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:19 smithi193 ceph-mon[35962]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:19.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:19 smithi150 ceph-mon[36204]: Deploying daemon mds.l on smithi193 2024-05-14T16:59:19.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:19 smithi150 ceph-mon[36204]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:20.541 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 systemd[1]: Starting Ceph mds.l for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 -- 172.21.15.99:0/1477532993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901088f0 msgr2=0x7f5690110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 --2- 172.21.15.99:0/1477532993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901088f0 0x7f5690110470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f568800b0d0 tx=0x7f568800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 -- 172.21.15.99:0/1477532993 shutdown_connections 2024-05-14T16:59:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 --2- 172.21.15.99:0/1477532993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901088f0 0x7f5690110470 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 --2- 172.21.15.99:0/1477532993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56901037f0 0x7f5690103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 --2- 172.21.15.99:0/1477532993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56901041a0 0x7f56901081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 -- 172.21.15.99:0/1477532993 >> 172.21.15.99:0/1477532993 conn(0x7f569007a6a0 msgr2=0x7f569007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:20.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 -- 172.21.15.99:0/1477532993 shutdown_connections 2024-05-14T16:59:20.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.636+0000 7f5697fe1700 1 -- 172.21.15.99:0/1477532993 wait complete. 2024-05-14T16:59:20.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 Processor -- start 2024-05-14T16:59:20.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 -- start start 2024-05-14T16:59:20.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56901037f0 0x7f569019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:20.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 0x7f569019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:20.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56901088f0 0x7f5690199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:20.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56901137b0 con 0x7f56901041a0 2024-05-14T16:59:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5690113630 con 0x7f56901088f0 2024-05-14T16:59:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f5697fe1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5690113930 con 0x7f56901037f0 2024-05-14T16:59:20.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.637+0000 7f569557c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 0x7f569019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:20.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 0x7f569019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54982/0 (socket says 172.21.15.99:54982) 2024-05-14T16:59:20.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 -- 172.21.15.99:0/3545672163 learned_addr learned my addr 172.21.15.99:0/3545672163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:20.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f5695d7d700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56901037f0 0x7f569019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569657e700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56901088f0 0x7f5690199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 -- 172.21.15.99:0/3545672163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56901037f0 msgr2=0x7f569019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56901037f0 0x7f569019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 -- 172.21.15.99:0/3545672163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56901088f0 msgr2=0x7f5690199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56901088f0 0x7f5690199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 -- 172.21.15.99:0/3545672163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569019a1b0 con 0x7f56901041a0 2024-05-14T16:59:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f569557c700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 0x7f569019fdd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f568000b530 tx=0x7f568000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:21.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680015020 con 0x7f56901041a0 2024-05-14T16:59:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569019a4a0 con 0x7f56901041a0 2024-05-14T16:59:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5680011ce0 con 0x7f56901041a0 2024-05-14T16:59:21.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680010dd0 con 0x7f56901041a0 2024-05-14T16:59:21.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.638+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a6760 con 0x7f56901041a0 2024-05-14T16:59:21.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.639+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569006a230 con 0x7f56901041a0 2024-05-14T16:59:21.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.640+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5680018020 con 0x7f56901041a0 2024-05-14T16:59:21.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.641+0000 7f5686ffd700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f567c0b2880 0x7f567c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.641+0000 7f5695d7d700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f567c0b2880 0x7f567c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.642+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f56800e1430 con 0x7f56901041a0 2024-05-14T16:59:21.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.642+0000 7f5695d7d700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f567c0b2880 0x7f567c0b4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f569019ab90 tx=0x7f568c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.646+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f568001f020 con 0x7f56901041a0 2024-05-14T16:59:21.243 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 podman[66748]: 2024-05-14 16:59:20.541706038 +0000 UTC m=+0.087892773 container create 291487fb2beb0c51269a54362b0064a46dae52a9041599588367e3734f57b3dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-14T16:59:21.250 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 podman[66748]: 2024-05-14 16:59:20.470550851 +0000 UTC m=+0.016737598 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:21.256 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 podman[66748]: 2024-05-14 16:59:20.727589555 +0000 UTC m=+0.273776290 container init 291487fb2beb0c51269a54362b0064a46dae52a9041599588367e3734f57b3dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-14T16:59:21.263 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 podman[66748]: 2024-05-14 16:59:20.730295297 +0000 UTC m=+0.276482033 container start 291487fb2beb0c51269a54362b0064a46dae52a9041599588367e3734f57b3dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-14T16:59:21.286 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 ceph-mds[66767]: -- 172.21.15.193:0/539047925 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55787831fa40 con 0x5578782f2c00 2024-05-14T16:59:21.294 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l[66763]: starting mds.l at 2024-05-14T16:59:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.909+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f569002e560 con 0x7f56901041a0 2024-05-14T16:59:21.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.910+0000 7f5686ffd700 1 -- 172.21.15.99:0/3545672163 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f56800a95f0 con 0x7f56901041a0 2024-05-14T16:59:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:21.353 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T16:59:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f567c0b2880 msgr2=0x7f567c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f567c0b2880 0x7f567c0b4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f569019ab90 tx=0x7f568c009190 comp rx=0 tx=0).stop 2024-05-14T16:59:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 msgr2=0x7f569019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 0x7f569019fdd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f568000b530 tx=0x7f568000b840 comp rx=0 tx=0).stop 2024-05-14T16:59:21.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 shutdown_connections 2024-05-14T16:59:21.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f567c0b2880 0x7f567c0b4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56901088f0 0x7f5690199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56901037f0 0x7f569019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 --2- 172.21.15.99:0/3545672163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56901041a0 0x7f569019fdd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.913+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 >> 172.21.15.99:0/3545672163 conn(0x7f569007a6a0 msgr2=0x7f5690101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:21.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.914+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 shutdown_connections 2024-05-14T16:59:21.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:20.914+0000 7f5697fe1700 1 -- 172.21.15.99:0/3545672163 wait complete. 2024-05-14T16:59:21.460 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2024-05-14T16:59:21.479 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 bash[66748]: 291487fb2beb0c51269a54362b0064a46dae52a9041599588367e3734f57b3dd 2024-05-14T16:59:21.494 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 16:59:20 smithi193 systemd[1]: Started Ceph mds.l for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:22.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:22.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6832/3825469703,v1:172.21.15.193:6833/3825469703] up:boot 2024-05-14T16:59:22.040 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: fsmap 1 up:standby 2024-05-14T16:59:22.048 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:59:22.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.070 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.081 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.090 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:22.114 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:22.122 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:22.130 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: Deploying daemon mds.k on smithi150 2024-05-14T16:59:22.136 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:21 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3545672163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:22.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:22.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6832/3825469703,v1:172.21.15.193:6833/3825469703] up:boot 2024-05-14T16:59:22.160 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: fsmap 1 up:standby 2024-05-14T16:59:22.200 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:59:22.219 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.234 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.242 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:22.249 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:22.257 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:22.272 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: Deploying daemon mds.k on smithi150 2024-05-14T16:59:22.279 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:21 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3545672163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:22.288 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:22.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6832/3825469703,v1:172.21.15.193:6833/3825469703] up:boot 2024-05-14T16:59:22.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: fsmap 1 up:standby 2024-05-14T16:59:22.322 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:59:22.330 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.339 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.347 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:22.355 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:22.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:22.399 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:22.407 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: Deploying daemon mds.k on smithi150 2024-05-14T16:59:22.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:21 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3545672163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:23.203 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 systemd[1]: Starting Ceph mds.k for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:23.491 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 podman[65659]: 2024-05-14 16:59:23.125976604 +0000 UTC m=+0.017559597 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:23.548 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 podman[65659]: 2024-05-14 16:59:23.238962425 +0000 UTC m=+0.130545412 container create 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T16:59:23.555 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 podman[65659]: 2024-05-14 16:59:23.466346584 +0000 UTC m=+0.357929577 container init 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:59:23.573 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 podman[65659]: 2024-05-14 16:59:23.468959244 +0000 UTC m=+0.360542239 container start 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:59:23.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:23.764 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mds[65677]: -- 172.21.15.150:0/1377579605 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55659bd6ba40 con 0x55659bd3e800 2024-05-14T16:59:23.777 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k[65673]: starting mds.k at 2024-05-14T16:59:23.789 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 bash[65659]: 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c 2024-05-14T16:59:23.801 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 16:59:23 smithi150 systemd[1]: Started Ceph mds.k for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:23.878 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:23.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:24.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.059 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.067 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:24.086 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:24.097 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:23 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:24.108 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:24.123 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.141 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.153 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:24.175 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:24.186 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:23 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:24.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:24.211 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:24.258 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:24.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:23 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:24.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:24 smithi150 ceph-mon[36204]: Deploying daemon mds.j on smithi099 2024-05-14T16:59:24.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:24 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6832/308217426,v1:172.21.15.150:6833/308217426] up:boot 2024-05-14T16:59:24.921 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:24 smithi150 ceph-mon[36204]: fsmap 2 up:standby 2024-05-14T16:59:24.930 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:24 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:59:25.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:24 smithi193 ceph-mon[35962]: Deploying daemon mds.j on smithi099 2024-05-14T16:59:25.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:24 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6832/308217426,v1:172.21.15.150:6833/308217426] up:boot 2024-05-14T16:59:25.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:24 smithi193 ceph-mon[35962]: fsmap 2 up:standby 2024-05-14T16:59:25.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:24 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:59:25.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:24 smithi099 ceph-mon[31851]: Deploying daemon mds.j on smithi099 2024-05-14T16:59:25.132 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:24 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6832/308217426,v1:172.21.15.150:6833/308217426] up:boot 2024-05-14T16:59:25.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:24 smithi099 ceph-mon[31851]: fsmap 2 up:standby 2024-05-14T16:59:25.155 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:24 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:59:25.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 -- 172.21.15.99:0/2231368139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 msgr2=0x7f621c10eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 --2- 172.21.15.99:0/2231368139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c10eb40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f621400c210 tx=0x7f621400c520 comp rx=0 tx=0).stop 2024-05-14T16:59:25.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 -- 172.21.15.99:0/2231368139 shutdown_connections 2024-05-14T16:59:25.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 --2- 172.21.15.99:0/2231368139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c10eb40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 --2- 172.21.15.99:0/2231368139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621c102160 0x7f621c102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 --2- 172.21.15.99:0/2231368139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c102b10 0x7f621c1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 -- 172.21.15.99:0/2231368139 >> 172.21.15.99:0/2231368139 conn(0x7f621c077780 msgr2=0x7f621c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:25.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.491+0000 7f62215a0700 1 -- 172.21.15.99:0/2231368139 shutdown_connections 2024-05-14T16:59:25.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.492+0000 7f62215a0700 1 -- 172.21.15.99:0/2231368139 wait complete. 2024-05-14T16:59:25.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.492+0000 7f62215a0700 1 Processor -- start 2024-05-14T16:59:25.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.492+0000 7f62215a0700 1 -- start start 2024-05-14T16:59:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.492+0000 7f62215a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c102160 0x7f621c103900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.492+0000 7f62215a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621c102b10 0x7f621c103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f62215a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:25.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f62215a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f621c1115b0 con 0x7f621c107000 2024-05-14T16:59:25.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f62215a0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f621c111430 con 0x7f621c102b10 2024-05-14T16:59:25.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f62215a0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f621c111730 con 0x7f621c102160 2024-05-14T16:59:25.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44242/0 (socket says 172.21.15.99:44242) 2024-05-14T16:59:25.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 -- 172.21.15.99:0/581424449 learned_addr learned my addr 172.21.15.99:0/581424449 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:25.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621a7fc700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621c102b10 0x7f621c103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621affd700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c102160 0x7f621c103900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:25.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 -- 172.21.15.99:0/581424449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c102160 msgr2=0x7f621c103900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c102160 0x7f621c103900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 -- 172.21.15.99:0/581424449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621c102b10 msgr2=0x7f621c103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:25.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621c102b10 0x7f621c103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:25.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 -- 172.21.15.99:0/581424449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c1a8140 con 0x7f621c107000 2024-05-14T16:59:25.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f621b7fe700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c104380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6214004fd0 tx=0x7f621400a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:25.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214022020 con 0x7f621c107000 2024-05-14T16:59:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c1a83d0 con 0x7f621c107000 2024-05-14T16:59:25.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6214008d30 con 0x7f621c107000 2024-05-14T16:59:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214007e30 con 0x7f621c107000 2024-05-14T16:59:25.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.493+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c1a8880 con 0x7f621c107000 2024-05-14T16:59:25.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.494+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c06a230 con 0x7f621c107000 2024-05-14T16:59:25.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.496+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6214025020 con 0x7f621c107000 2024-05-14T16:59:25.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.497+0000 7f6203fff700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f62040b2880 0x7f62040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.497+0000 7f621affd700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f62040b2880 0x7f62040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:26.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.497+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f62140e3480 con 0x7f621c107000 2024-05-14T16:59:26.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.497+0000 7f621affd700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f62040b2880 0x7f62040b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.500+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f621402c020 con 0x7f621c107000 2024-05-14T16:59:26.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:25 smithi193 ceph-mon[35962]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:26.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:25 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/581424449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:26.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.762+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f621c02e680 con 0x7f621c107000 2024-05-14T16:59:26.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.763+0000 7f6203fff700 1 -- 172.21.15.99:0/581424449 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f62140ab6d0 con 0x7f621c107000 2024-05-14T16:59:26.104 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:26.114 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T16:59:26.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f62040b2880 msgr2=0x7f62040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:26.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f62040b2880 0x7f62040b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c005f60 comp rx=0 tx=0).stop 2024-05-14T16:59:26.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 msgr2=0x7f621c104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:26.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c104380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6214004fd0 tx=0x7f621400a910 comp rx=0 tx=0).stop 2024-05-14T16:59:26.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 shutdown_connections 2024-05-14T16:59:26.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f62040b2880 0x7f62040b4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f621c107000 0x7f621c104380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c102160 0x7f621c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 --2- 172.21.15.99:0/581424449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621c102b10 0x7f621c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:26.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 >> 172.21.15.99:0/581424449 conn(0x7f621c077780 msgr2=0x7f621c100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:26.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 shutdown_connections 2024-05-14T16:59:26.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:25.765+0000 7f62215a0700 1 -- 172.21.15.99:0/581424449 wait complete. 2024-05-14T16:59:26.335 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2024-05-14T16:59:26.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:25 smithi099 ceph-mon[31851]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:26.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:25 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/581424449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:26.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:25 smithi150 ceph-mon[36204]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:26.377 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:25 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/581424449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:28.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:27 smithi099 ceph-mon[31851]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:28.028 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:27 smithi099 systemd[1]: Starting Ceph mds.j for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:28.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:27 smithi193 ceph-mon[35962]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:28.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:27 smithi150 ceph-mon[36204]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:28.341 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 podman[70022]: 2024-05-14 16:59:27.986740464 +0000 UTC m=+0.017622975 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:28.363 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 podman[70022]: 2024-05-14 16:59:28.106530799 +0000 UTC m=+0.137413310 container create 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-14T16:59:28.373 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 podman[70022]: 2024-05-14 16:59:28.342103168 +0000 UTC m=+0.372985675 container init 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5) 2024-05-14T16:59:28.764 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 podman[70022]: 2024-05-14 16:59:28.345062445 +0000 UTC m=+0.375944962 container start 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-14T16:59:28.778 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 ceph-mds[70042]: -- 172.21.15.99:0/4260188244 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561f1699a40 con 0x5561f166cc00 2024-05-14T16:59:28.790 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j[70038]: starting mds.j at 2024-05-14T16:59:28.802 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 bash[70022]: 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee 2024-05-14T16:59:28.812 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 16:59:28 smithi099 systemd[1]: Started Ceph mds.j for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:28.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:29.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:29.772 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:29.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:29.789 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:29.795 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:29.801 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:29.807 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:29.813 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: Deploying daemon mds.d on smithi099 2024-05-14T16:59:29.819 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.99:6834/3118783767,v1:172.21.15.99:6835/3118783767] up:boot 2024-05-14T16:59:29.825 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: fsmap 3 up:standby 2024-05-14T16:59:29.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:29 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:59:29.876 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:29.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:29.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:29.938 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:29.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:29.950 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:29.957 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:29.964 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: Deploying daemon mds.d on smithi099 2024-05-14T16:59:29.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.99:6834/3118783767,v1:172.21.15.99:6835/3118783767] up:boot 2024-05-14T16:59:29.976 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: fsmap 3 up:standby 2024-05-14T16:59:29.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:29 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:59:29.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:30.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:30.007 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:30.014 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:30.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:30.027 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:30.042 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:30.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: Deploying daemon mds.d on smithi099 2024-05-14T16:59:30.091 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.99:6834/3118783767,v1:172.21.15.99:6835/3118783767] up:boot 2024-05-14T16:59:30.097 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: fsmap 3 up:standby 2024-05-14T16:59:30.105 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:29 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:59:30.216 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:30.764 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:30 smithi099 systemd[1]: Starting Ceph mds.d for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:31.769 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:31 smithi099 ceph-mon[31851]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:32.512 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 podman[70420]: 2024-05-14 16:59:31.450024345 +0000 UTC m=+0.018454702 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:32.538 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 podman[70420]: 2024-05-14 16:59:31.550826417 +0000 UTC m=+0.119256746 container create d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T16:59:32.547 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 podman[70420]: 2024-05-14 16:59:31.778651841 +0000 UTC m=+0.347082186 container init d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-14T16:59:32.562 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 podman[70420]: 2024-05-14 16:59:31.781499895 +0000 UTC m=+0.349930232 container start d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-14T16:59:32.573 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 ceph-mds[70457]: -- 172.21.15.99:0/2342646168 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555defd9fa40 con 0x555defd73000 2024-05-14T16:59:32.579 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d[70453]: starting mds.d at 2024-05-14T16:59:32.586 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 bash[70420]: d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc 2024-05-14T16:59:32.592 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 16:59:31 smithi099 systemd[1]: Started Ceph mds.d for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:32.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:31 smithi193 ceph-mon[35962]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:32.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:31 smithi150 ceph-mon[36204]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:32.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 -- 172.21.15.99:0/125463695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80108520 msgr2=0x7f8e80108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:32.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 --2- 172.21.15.99:0/125463695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80108520 0x7f8e80108900 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009990 tx=0x7f8e78009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 -- 172.21.15.99:0/125463695 shutdown_connections 2024-05-14T16:59:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 --2- 172.21.15.99:0/125463695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e80102e00 0x7f8e8010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:32.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 --2- 172.21.15.99:0/125463695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80108520 0x7f8e80108900 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:32.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 --2- 172.21.15.99:0/125463695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e80102440 0x7f8e801028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:32.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 -- 172.21.15.99:0/125463695 >> 172.21.15.99:0/125463695 conn(0x7f8e800793f0 msgr2=0x7f8e80079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 -- 172.21.15.99:0/125463695 shutdown_connections 2024-05-14T16:59:32.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.401+0000 7f8e87248700 1 -- 172.21.15.99:0/125463695 wait complete. 2024-05-14T16:59:32.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 Processor -- start 2024-05-14T16:59:32.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 -- start start 2024-05-14T16:59:32.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 0x7f8e80110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:32.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e80102e00 0x7f8e80110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:32.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e80108520 0x7f8e80110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:32.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801136e0 con 0x7f8e80102440 2024-05-14T16:59:32.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80113560 con 0x7f8e80108520 2024-05-14T16:59:32.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e87248700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80113860 con 0x7f8e80102e00 2024-05-14T16:59:32.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e84fe4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 0x7f8e80110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:32.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e84fe4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 0x7f8e80110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44292/0 (socket says 172.21.15.99:44292) 2024-05-14T16:59:32.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e84fe4700 1 -- 172.21.15.99:0/371608604 learned_addr learned my addr 172.21.15.99:0/371608604 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:32.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.402+0000 7f8e857e5700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e80108520 0x7f8e80110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:32.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e7ffff700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e80102e00 0x7f8e80110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:32.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e84fe4700 1 -- 172.21.15.99:0/371608604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e80102e00 msgr2=0x7f8e80110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:32.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e84fe4700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e80102e00 0x7f8e80110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:32.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e84fe4700 1 -- 172.21.15.99:0/371608604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e80108520 msgr2=0x7f8e80110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e84fe4700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e80108520 0x7f8e80110a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e84fe4700 1 -- 172.21.15.99:0/371608604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e801a81e0 con 0x7f8e80102440 2024-05-14T16:59:33.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e84fe4700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 0x7f8e80110010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e78000c00 tx=0x7f8e780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:33.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78027070 con 0x7f8e80102440 2024-05-14T16:59:33.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e801a8470 con 0x7f8e80102440 2024-05-14T16:59:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e78003fa0 con 0x7f8e80102440 2024-05-14T16:59:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78022d40 con 0x7f8e80102440 2024-05-14T16:59:33.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.403+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e801a8920 con 0x7f8e80102440 2024-05-14T16:59:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.404+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8010a9b0 con 0x7f8e80102440 2024-05-14T16:59:33.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.406+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8e7802a070 con 0x7f8e80102440 2024-05-14T16:59:33.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.406+0000 7f8e7dffb700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f8e6c0b2880 0x7f8e6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:33.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.406+0000 7f8e7ffff700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f8e6c0b2880 0x7f8e6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:33.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.407+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8e780e2e10 con 0x7f8e80102440 2024-05-14T16:59:33.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.407+0000 7f8e7ffff700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f8e6c0b2880 0x7f8e6c0b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e80079b10 tx=0x7f8e70007790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:33.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.410+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e78031070 con 0x7f8e80102440 2024-05-14T16:59:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.662+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e8006a230 con 0x7f8e80102440 2024-05-14T16:59:33.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.662+0000 7f8e7dffb700 1 -- 172.21.15.99:0/371608604 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f8e78013070 con 0x7f8e80102440 2024-05-14T16:59:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:33.188 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T16:59:33.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f8e6c0b2880 msgr2=0x7f8e6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:33.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f8e6c0b2880 0x7f8e6c0b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e80079b10 tx=0x7f8e70007790 comp rx=0 tx=0).stop 2024-05-14T16:59:33.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 msgr2=0x7f8e80110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:33.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 0x7f8e80110010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e78000c00 tx=0x7f8e780047b0 comp rx=0 tx=0).stop 2024-05-14T16:59:33.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 shutdown_connections 2024-05-14T16:59:33.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f8e6c0b2880 0x7f8e6c0b4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:33.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e80108520 0x7f8e80110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:33.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e80102440 0x7f8e80110010 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 --2- 172.21.15.99:0/371608604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e80102e00 0x7f8e80110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 >> 172.21.15.99:0/371608604 conn(0x7f8e800793f0 msgr2=0x7f8e80101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:33.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 shutdown_connections 2024-05-14T16:59:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:32.665+0000 7f8e87248700 1 -- 172.21.15.99:0/371608604 wait complete. 2024-05-14T16:59:33.381 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 5 2024-05-14T16:59:33.390 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.396 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.403 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:33.416 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:33.423 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:33.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: Deploying daemon mds.e on smithi150 2024-05-14T16:59:33.436 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.99:6836/2633931884,v1:172.21.15.99:6837/2633931884] up:boot 2024-05-14T16:59:33.443 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: fsmap 4 up:standby 2024-05-14T16:59:33.449 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:59:33.469 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:32 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/371608604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:33.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.534 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:33.546 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:33.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:33.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: Deploying daemon mds.e on smithi150 2024-05-14T16:59:33.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.99:6836/2633931884,v1:172.21.15.99:6837/2633931884] up:boot 2024-05-14T16:59:33.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: fsmap 4 up:standby 2024-05-14T16:59:33.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:59:33.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:32 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/371608604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:33.597 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.603 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.610 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:33.616 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:33.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:33.643 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:33.668 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: Deploying daemon mds.e on smithi150 2024-05-14T16:59:33.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.99:6836/2633931884,v1:172.21.15.99:6837/2633931884] up:boot 2024-05-14T16:59:33.693 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: fsmap 4 up:standby 2024-05-14T16:59:33.700 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:59:33.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:32 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/371608604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:33.953 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:33 smithi150 systemd[1]: Starting Ceph mds.e for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:34.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:33 smithi150 ceph-mon[36204]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:34.228 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 podman[65972]: 2024-05-14 16:59:33.995515194 +0000 UTC m=+0.017076397 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:34.235 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 podman[65972]: 2024-05-14 16:59:34.099794503 +0000 UTC m=+0.121355682 container create c1b79095d2af9e2d651964d9f006c7af2ebb343e064c9e070440ec3135f7de7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:59:34.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:33 smithi099 ceph-mon[31851]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:34.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:33 smithi193 ceph-mon[35962]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:34.703 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 podman[65972]: 2024-05-14 16:59:34.361818208 +0000 UTC m=+0.383379394 container init c1b79095d2af9e2d651964d9f006c7af2ebb343e064c9e070440ec3135f7de7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:59:34.720 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 podman[65972]: 2024-05-14 16:59:34.364641446 +0000 UTC m=+0.386202633 container start c1b79095d2af9e2d651964d9f006c7af2ebb343e064c9e070440ec3135f7de7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-14T16:59:34.728 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 ceph-mds[65991]: -- 172.21.15.150:0/106014590 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623245c9a40 con 0x56232459c800 2024-05-14T16:59:34.734 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e[65987]: starting mds.e at 2024-05-14T16:59:34.740 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 bash[65972]: c1b79095d2af9e2d651964d9f006c7af2ebb343e064c9e070440ec3135f7de7d 2024-05-14T16:59:34.745 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 16:59:34 smithi150 systemd[1]: Started Ceph mds.e for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:35.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:35.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.718 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.731 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:35.746 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:35.753 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:35.759 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: Deploying daemon mds.f on smithi193 2024-05-14T16:59:35.766 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6834/955154875,v1:172.21.15.150:6835/955154875] up:boot 2024-05-14T16:59:35.775 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: fsmap 5 up:standby 2024-05-14T16:59:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:35 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:59:35.786 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:35.794 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:35.801 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.809 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.817 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.825 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:35.841 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:35.881 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:35.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: Deploying daemon mds.f on smithi193 2024-05-14T16:59:35.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6834/955154875,v1:172.21.15.150:6835/955154875] up:boot 2024-05-14T16:59:35.905 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: fsmap 5 up:standby 2024-05-14T16:59:35.911 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:35 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:59:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:35.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.939 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:35.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:35.955 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:35.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:35.968 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: Deploying daemon mds.f on smithi193 2024-05-14T16:59:35.973 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6834/955154875,v1:172.21.15.150:6835/955154875] up:boot 2024-05-14T16:59:35.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: fsmap 5 up:standby 2024-05-14T16:59:35.987 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:35 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:59:36.168 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:36.684 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 systemd[1]: Starting Ceph mds.f for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:37.003 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 podman[67054]: 2024-05-14 16:59:36.68458353 +0000 UTC m=+0.083834073 container create ed371fb0d113495117ef712b0cd532b3ad1132afa68c3bad9b77daf0afcd5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:59:37.011 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 podman[67054]: 2024-05-14 16:59:36.617725901 +0000 UTC m=+0.016976460 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:37.018 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 podman[67054]: 2024-05-14 16:59:36.904317194 +0000 UTC m=+0.303567737 container init ed371fb0d113495117ef712b0cd532b3ad1132afa68c3bad9b77daf0afcd5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T16:59:37.025 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 podman[67054]: 2024-05-14 16:59:36.907151703 +0000 UTC m=+0.306402256 container start ed371fb0d113495117ef712b0cd532b3ad1132afa68c3bad9b77daf0afcd5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-14T16:59:37.032 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 ceph-mds[67073]: -- 172.21.15.193:0/3080375527 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597854fba40 con 0x5597854ce800 2024-05-14T16:59:37.038 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: starting mds.f at 2024-05-14T16:59:37.044 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:37 smithi193 bash[67054]: ed371fb0d113495117ef712b0cd532b3ad1132afa68c3bad9b77daf0afcd5cb1 2024-05-14T16:59:37.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 16:59:37 smithi193 systemd[1]: Started Ceph mds.f for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 -- 172.21.15.99:0/1323772855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41037f0 msgr2=0x7feef4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 --2- 172.21.15.99:0/1323772855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41037f0 0x7feef4103bd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feef00099c0 tx=0x7feef0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:59:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 -- 172.21.15.99:0/1323772855 shutdown_connections 2024-05-14T16:59:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 --2- 172.21.15.99:0/1323772855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41088f0 0x7feef4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:37.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 --2- 172.21.15.99:0/1323772855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41037f0 0x7feef4103bd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:37.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 --2- 172.21.15.99:0/1323772855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41041a0 0x7feef41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:37.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 -- 172.21.15.99:0/1323772855 >> 172.21.15.99:0/1323772855 conn(0x7feef407a6a0 msgr2=0x7feef407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 -- 172.21.15.99:0/1323772855 shutdown_connections 2024-05-14T16:59:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.848+0000 7feefb8e9700 1 -- 172.21.15.99:0/1323772855 wait complete. 2024-05-14T16:59:38.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 Processor -- start 2024-05-14T16:59:38.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 -- start start 2024-05-14T16:59:38.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41037f0 0x7feef4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 0x7feef4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41088f0 0x7feef40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feef41136c0 con 0x7feef41088f0 2024-05-14T16:59:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feef4113540 con 0x7feef41041a0 2024-05-14T16:59:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feefb8e9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feef4113840 con 0x7feef41037f0 2024-05-14T16:59:38.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feef8e84700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 0x7feef4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feef9e86700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41088f0 0x7feef40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feef8e84700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 0x7feef4075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:51020/0 (socket says 172.21.15.99:51020) 2024-05-14T16:59:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.849+0000 7feef8e84700 1 -- 172.21.15.99:0/127261557 learned_addr learned my addr 172.21.15.99:0/127261557 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9685700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41037f0 0x7feef4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9e86700 1 -- 172.21.15.99:0/127261557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41037f0 msgr2=0x7feef4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9e86700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41037f0 0x7feef4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9e86700 1 -- 172.21.15.99:0/127261557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 msgr2=0x7feef4075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9e86700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 0x7feef4075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9e86700 1 -- 172.21.15.99:0/127261557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef41ac440 con 0x7feef41088f0 2024-05-14T16:59:38.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9685700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41037f0 0x7feef4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef8e84700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 0x7feef4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feef9e86700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41088f0 0x7feef40762c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feeec00cc50 tx=0x7feeec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:38.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec016020 con 0x7feef41088f0 2024-05-14T16:59:38.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef41ac6d0 con 0x7feef41088f0 2024-05-14T16:59:38.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeec012420 con 0x7feef41088f0 2024-05-14T16:59:38.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec0115a0 con 0x7feef41088f0 2024-05-14T16:59:38.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.850+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef41acbc0 con 0x7feef41088f0 2024-05-14T16:59:38.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.851+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef410bb30 con 0x7feef41088f0 2024-05-14T16:59:38.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.853+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7feeec019020 con 0x7feef41088f0 2024-05-14T16:59:38.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.854+0000 7feeea7fc700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feee00b2880 0x7feee00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:38.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.854+0000 7feef9685700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feee00b2880 0x7feee00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:38.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.854+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7feeec0e1f90 con 0x7feef41088f0 2024-05-14T16:59:38.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.854+0000 7feef9685700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feee00b2880 0x7feee00b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feef0009990 tx=0x7feef0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:38.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:37.858+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feeec020020 con 0x7feef41088f0 2024-05-14T16:59:38.422 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:38.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.449 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.455 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.461 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:38.467 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:38.473 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:38.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: Deploying daemon mds.c on smithi193 2024-05-14T16:59:38.485 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6834/3113222034,v1:172.21.15.193:6835/3113222034] up:boot 2024-05-14T16:59:38.491 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: fsmap 6 up:standby 2024-05-14T16:59:38.497 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:38 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:59:38.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.107+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feef4104c80 con 0x7feef41088f0 2024-05-14T16:59:38.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:38.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.527 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.542 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:38.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:38.599 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:38.638 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: Deploying daemon mds.c on smithi193 2024-05-14T16:59:38.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6834/3113222034,v1:172.21.15.193:6835/3113222034] up:boot 2024-05-14T16:59:38.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: fsmap 6 up:standby 2024-05-14T16:59:38.658 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:38 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:59:38.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.107+0000 7feeea7fc700 1 -- 172.21.15.99:0/127261557 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7feeec0aa150 con 0x7feef41088f0 2024-05-14T16:59:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:38.676 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-14T16:59:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feee00b2880 msgr2=0x7feee00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feee00b2880 0x7feee00b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feef0009990 tx=0x7feef0000bc0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41088f0 msgr2=0x7feef40762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:38.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41088f0 0x7feef40762c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feeec00cc50 tx=0x7feeec00cf60 comp rx=0 tx=0).stop 2024-05-14T16:59:38.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 shutdown_connections 2024-05-14T16:59:38.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7feee00b2880 0x7feee00b4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feef41088f0 0x7feef40762c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef41037f0 0x7feef4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 --2- 172.21.15.99:0/127261557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feef41041a0 0x7feef4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 >> 172.21.15.99:0/127261557 conn(0x7feef407a6a0 msgr2=0x7feef41015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 shutdown_connections 2024-05-14T16:59:38.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:38.110+0000 7feefb8e9700 1 -- 172.21.15.99:0/127261557 wait complete. 2024-05-14T16:59:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 7 2024-05-14T16:59:38.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:38.865 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.871 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.878 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:38.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:38.890 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:38.896 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:38.902 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: Deploying daemon mds.c on smithi193 2024-05-14T16:59:38.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6834/3113222034,v1:172.21.15.193:6835/3113222034] up:boot 2024-05-14T16:59:38.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: fsmap 6 up:standby 2024-05-14T16:59:38.920 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:38 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:59:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:39 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/127261557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:39.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:39 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/127261557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:39.333 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 systemd[1]: Starting Ceph mds.c for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:39 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/127261557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:39.565 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 podman[67359]: 2024-05-14 16:59:39.213856089 +0000 UTC m=+0.017306943 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:39.571 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 podman[67359]: 2024-05-14 16:59:39.346327242 +0000 UTC m=+0.149778102 container create d455cbdc36f6ec19e65b40187b0fe61b205a113a1f957904a2b0925fb9f8e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T16:59:39.577 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 podman[67359]: 2024-05-14 16:59:39.540660225 +0000 UTC m=+0.344111073 container init d455cbdc36f6ec19e65b40187b0fe61b205a113a1f957904a2b0925fb9f8e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T16:59:39.583 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 podman[67359]: 2024-05-14 16:59:39.543536105 +0000 UTC m=+0.346986944 container start d455cbdc36f6ec19e65b40187b0fe61b205a113a1f957904a2b0925fb9f8e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-14T16:59:39.589 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 ceph-mds[67378]: -- 172.21.15.193:0/2055320927 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee84aada40 con 0x55ee84a80800 2024-05-14T16:59:40.042 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c[67374]: starting mds.c at 2024-05-14T16:59:40.059 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 bash[67359]: d455cbdc36f6ec19e65b40187b0fe61b205a113a1f957904a2b0925fb9f8e70c 2024-05-14T16:59:40.067 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 16:59:39 smithi193 systemd[1]: Started Ceph mds.c for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:40.321 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:40.342 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.351 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.361 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:40.402 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:40.415 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:40 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:40.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:40.527 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.537 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.549 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.561 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:40.573 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:40.585 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:40 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:40.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:40.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.633 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:40.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:40.691 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:40.713 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:40 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:40.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:41.208 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:41 smithi099 ceph-mon[31851]: Deploying daemon mds.h on smithi150 2024-05-14T16:59:41.275 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:41 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] up:boot 2024-05-14T16:59:41.284 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:41 smithi099 ceph-mon[31851]: fsmap 7 up:standby 2024-05-14T16:59:41.291 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:41 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:59:41.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:41 smithi150 ceph-mon[36204]: Deploying daemon mds.h on smithi150 2024-05-14T16:59:41.352 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:41 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] up:boot 2024-05-14T16:59:41.361 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:41 smithi150 ceph-mon[36204]: fsmap 7 up:standby 2024-05-14T16:59:41.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:41 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:59:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:41 smithi193 ceph-mon[35962]: Deploying daemon mds.h on smithi150 2024-05-14T16:59:41.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:41 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] up:boot 2024-05-14T16:59:41.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:41 smithi193 ceph-mon[35962]: fsmap 7 up:standby 2024-05-14T16:59:41.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:41 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:59:42.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:42 smithi099 ceph-mon[31851]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:42.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:42 smithi193 ceph-mon[35962]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:42.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:42 smithi150 ceph-mon[36204]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:42.466 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 16:59:42 smithi150 systemd[1]: Started Ceph mds.h for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:42.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3446248051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec0104ff0 msgr2=0x7f5ec010d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:42.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3446248051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec0104ff0 0x7f5ec010d4e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3446248051 shutdown_connections 2024-05-14T16:59:42.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3446248051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec010dab0 0x7f5ec010ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3446248051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec01046d0 0x7f5ec0104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3446248051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec0104ff0 0x7f5ec010d4e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3446248051 >> 172.21.15.99:0/3446248051 conn(0x7f5ec00fde60 msgr2=0x7f5ec0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:42.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3446248051 shutdown_connections 2024-05-14T16:59:42.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.863+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3446248051 wait complete. 2024-05-14T16:59:42.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.864+0000 7f5ec4ba0700 1 Processor -- start 2024-05-14T16:59:42.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.864+0000 7f5ec4ba0700 1 -- start start 2024-05-14T16:59:43.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 0x7f5ec019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec0104ff0 0x7f5ec019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec010dab0 0x7f5ec0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0113640 con 0x7f5ec01046d0 2024-05-14T16:59:43.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01134c0 con 0x7f5ec0104ff0 2024-05-14T16:59:43.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 0x7f5ec019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01137c0 con 0x7f5ec010dab0 2024-05-14T16:59:43.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 0x7f5ec019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53946/0 (socket says 172.21.15.99:53946) 2024-05-14T16:59:43.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 -- 172.21.15.99:0/3496536658 learned_addr learned my addr 172.21.15.99:0/3496536658 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:43.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebffff700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec010dab0 0x7f5ec0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5eb7fff700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec0104ff0 0x7f5ec019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 -- 172.21.15.99:0/3496536658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec010dab0 msgr2=0x7f5ec0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:43.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec010dab0 0x7f5ec0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 -- 172.21.15.99:0/3496536658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec0104ff0 msgr2=0x7f5ec019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:43.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec0104ff0 0x7f5ec019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 -- 172.21.15.99:0/3496536658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec019a0e0 con 0x7f5ec01046d0 2024-05-14T16:59:43.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5eb7fff700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec0104ff0 0x7f5ec019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:43.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebf7fe700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 0x7f5ec019f7c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000c920 tx=0x7f5eb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:43.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0007b00 con 0x7f5ec01046d0 2024-05-14T16:59:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec019a3d0 con 0x7f5ec01046d0 2024-05-14T16:59:43.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.865+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb0004ce0 con 0x7f5ec01046d0 2024-05-14T16:59:43.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.866+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0005500 con 0x7f5ec01046d0 2024-05-14T16:59:43.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.866+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01a6690 con 0x7f5ec01046d0 2024-05-14T16:59:43.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.867+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec019aeb0 con 0x7f5ec01046d0 2024-05-14T16:59:43.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.868+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5eb0007ca0 con 0x7f5ec01046d0 2024-05-14T16:59:43.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.868+0000 7f5ebd7fa700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5eac0b2880 0x7f5eac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.869+0000 7f5eb7fff700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5eac0b2880 0x7f5eac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.869+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f5eb00e1d00 con 0x7f5ec01046d0 2024-05-14T16:59:43.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.869+0000 7f5eb7fff700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5eac0b2880 0x7f5eac0b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8006010 tx=0x7f5ea801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:43.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:42.872+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5eb0014030 con 0x7f5ec01046d0 2024-05-14T16:59:43.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.128+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec006a210 con 0x7f5ec01046d0 2024-05-14T16:59:43.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.128+0000 7f5ebd7fa700 1 -- 172.21.15.99:0/3496536658 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f5eb00a9ec0 con 0x7f5ec01046d0 2024-05-14T16:59:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:43.493 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-14T16:59:43.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.130+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5eac0b2880 msgr2=0x7f5eac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:43.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.130+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5eac0b2880 0x7f5eac0b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8006010 tx=0x7f5ea801a040 comp rx=0 tx=0).stop 2024-05-14T16:59:43.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.130+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 msgr2=0x7f5ec019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:43.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.130+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 0x7f5ec019f7c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000c920 tx=0x7f5eb000cc30 comp rx=0 tx=0).stop 2024-05-14T16:59:43.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 shutdown_connections 2024-05-14T16:59:43.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5eac0b2880 0x7f5eac0b4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec010dab0 0x7f5ec0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ec01046d0 0x7f5ec019f7c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 --2- 172.21.15.99:0/3496536658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec0104ff0 0x7f5ec019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 >> 172.21.15.99:0/3496536658 conn(0x7f5ec00fde60 msgr2=0x7f5ec00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:43.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 shutdown_connections 2024-05-14T16:59:43.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:43.131+0000 7f5ec4ba0700 1 -- 172.21.15.99:0/3496536658 wait complete. 2024-05-14T16:59:43.608 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 9 2024-05-14T16:59:43.615 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.621 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.628 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.634 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:43.643 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:43.651 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:43.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: Deploying daemon mds.i on smithi193 2024-05-14T16:59:43.721 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:43.732 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6836/1891176204,v1:172.21.15.150:6837/1891176204] up:boot 2024-05-14T16:59:43.743 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: fsmap 8 up:standby 2024-05-14T16:59:43.773 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:59:43.783 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:43 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3496536658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:43.792 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.803 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.815 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.827 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:43.843 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:43.853 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:43.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: Deploying daemon mds.i on smithi193 2024-05-14T16:59:43.869 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:43.877 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6836/1891176204,v1:172.21.15.150:6837/1891176204] up:boot 2024-05-14T16:59:43.887 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: fsmap 8 up:standby 2024-05-14T16:59:43.895 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:59:43.931 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:43 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3496536658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:43.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:43.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:44.007 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:44.034 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:44.043 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:44.063 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: Deploying daemon mds.i on smithi193 2024-05-14T16:59:44.070 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:44.078 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6836/1891176204,v1:172.21.15.150:6837/1891176204] up:boot 2024-05-14T16:59:44.084 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: fsmap 8 up:standby 2024-05-14T16:59:44.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:59:44.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:43 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3496536658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:44.721 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 systemd[1]: Starting Ceph mds.i for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:45.042 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 podman[67675]: 2024-05-14 16:59:44.624107061 +0000 UTC m=+0.017509613 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:45.059 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 podman[67675]: 2024-05-14 16:59:44.723991578 +0000 UTC m=+0.117394125 container create 3d536fb5be4615227372857796d43d297f6cb82f524e6dfbafb05bf8834494af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:59:45.074 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 podman[67675]: 2024-05-14 16:59:44.943225245 +0000 UTC m=+0.336627783 container init 3d536fb5be4615227372857796d43d297f6cb82f524e6dfbafb05bf8834494af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-14T16:59:45.135 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 podman[67675]: 2024-05-14 16:59:44.946087054 +0000 UTC m=+0.339489593 container start 3d536fb5be4615227372857796d43d297f6cb82f524e6dfbafb05bf8834494af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-14T16:59:45.189 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 ceph-mds[67694]: -- 172.21.15.193:0/2897421786 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee2f56ba40 con 0x55ee2f53ec00 2024-05-14T16:59:45.207 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: starting mds.i at 2024-05-14T16:59:45.542 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:45 smithi193 bash[67675]: 3d536fb5be4615227372857796d43d297f6cb82f524e6dfbafb05bf8834494af 2024-05-14T16:59:45.559 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 16:59:45 smithi193 systemd[1]: Started Ceph mds.i for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:46.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:46.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:46.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.504 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.513 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:46.526 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:46.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:46.547 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: Deploying daemon mds.g on smithi099 2024-05-14T16:59:46.558 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] up:boot 2024-05-14T16:59:46.583 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: fsmap 9 up:standby 2024-05-14T16:59:46.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:46 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:59:46.640 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:46.649 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.659 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:46.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:46.725 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:46.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: Deploying daemon mds.g on smithi099 2024-05-14T16:59:46.752 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] up:boot 2024-05-14T16:59:46.761 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: fsmap 9 up:standby 2024-05-14T16:59:46.769 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:46 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:59:46.778 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:46.789 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.796 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.804 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:46.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:46.824 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:46.847 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:46.858 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: Deploying daemon mds.g on smithi099 2024-05-14T16:59:46.870 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] up:boot 2024-05-14T16:59:46.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: fsmap 9 up:standby 2024-05-14T16:59:46.925 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:46 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:59:46.933 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:47.515 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:47 smithi099 systemd[1]: Starting Ceph mds.g for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:48.356 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:48 smithi099 ceph-mon[31851]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:48.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:48 smithi150 ceph-mon[36204]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:48.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 -- 172.21.15.99:0/3385881053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 msgr2=0x7f85ec1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3385881053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec1028c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009990 tx=0x7f85dc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 -- 172.21.15.99:0/3385881053 shutdown_connections 2024-05-14T16:59:48.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3385881053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 0x7f85ec10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3385881053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 0x7f85ec108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3385881053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec1028c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 -- 172.21.15.99:0/3385881053 >> 172.21.15.99:0/3385881053 conn(0x7f85ec0793f0 msgr2=0x7f85ec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:48.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 -- 172.21.15.99:0/3385881053 shutdown_connections 2024-05-14T16:59:48.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.460+0000 7f85f26c0700 1 -- 172.21.15.99:0/3385881053 wait complete. 2024-05-14T16:59:48.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 Processor -- start 2024-05-14T16:59:48.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 -- start start 2024-05-14T16:59:48.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 0x7f85ec19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 0x7f85ec1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1125d0 con 0x7f85ec102440 2024-05-14T16:59:48.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112450 con 0x7f85ec108520 2024-05-14T16:59:48.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85f26c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112750 con 0x7f85ec102e00 2024-05-14T16:59:48.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35228/0 (socket says 172.21.15.99:35228) 2024-05-14T16:59:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.461+0000 7f85ebfff700 1 -- 172.21.15.99:0/3649234881 learned_addr learned my addr 172.21.15.99:0/3649234881 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85f0c5d700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 0x7f85ec1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85eb7fe700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 0x7f85ec19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85ebfff700 1 -- 172.21.15.99:0/3649234881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 msgr2=0x7f85ec19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85ebfff700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 0x7f85ec19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85ebfff700 1 -- 172.21.15.99:0/3649234881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 msgr2=0x7f85ec1a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:48.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85ebfff700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 0x7f85ec1a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85ebfff700 1 -- 172.21.15.99:0/3649234881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec1a5f30 con 0x7f85ec102440 2024-05-14T16:59:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85f0c5d700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 0x7f85ec1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:48.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85eb7fe700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 0x7f85ec19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:48.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85ebfff700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec19a970 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c9b0 tx=0x7f85d400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:48.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007b00 con 0x7f85ec102440 2024-05-14T16:59:48.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec1a6160 con 0x7f85ec102440 2024-05-14T16:59:48.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d4007ca0 con 0x7f85ec102440 2024-05-14T16:59:48.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018730 con 0x7f85ec102440 2024-05-14T16:59:48.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.462+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a66c0 con 0x7f85ec102440 2024-05-14T16:59:49.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.463+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec06a230 con 0x7f85ec102440 2024-05-14T16:59:49.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.465+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f85d400f450 con 0x7f85ec102440 2024-05-14T16:59:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.466+0000 7f85e97fa700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f85d80b2880 0x7f85d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.466+0000 7f85eb7fe700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f85d80b2880 0x7f85d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.466+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f85d40e2040 con 0x7f85ec102440 2024-05-14T16:59:49.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.466+0000 7f85eb7fe700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f85d80b2880 0x7f85d80b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b5c0 tx=0x7f85dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:49.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.470+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85d4014030 con 0x7f85ec102440 2024-05-14T16:59:49.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:48 smithi193 ceph-mon[35962]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:49.104 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:48 smithi099 podman[71246]: 2024-05-14 16:59:48.495284037 +0000 UTC m=+0.017823115 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:49.113 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:48 smithi099 podman[71246]: 2024-05-14 16:59:48.62496098 +0000 UTC m=+0.147500055 container create c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True) 2024-05-14T16:59:49.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.735+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85ec02e560 con 0x7f85ec102440 2024-05-14T16:59:49.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.736+0000 7f85e97fa700 1 -- 172.21.15.99:0/3649234881 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f85d40aa290 con 0x7f85ec102440 2024-05-14T16:59:49.168 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:49.177 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-14T16:59:49.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f85d80b2880 msgr2=0x7f85d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:49.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f85d80b2880 0x7f85d80b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b5c0 tx=0x7f85dc005fb0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 msgr2=0x7f85ec19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec19a970 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c9b0 tx=0x7f85d400cd70 comp rx=0 tx=0).stop 2024-05-14T16:59:49.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 shutdown_connections 2024-05-14T16:59:49.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f85d80b2880 0x7f85d80b4d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85ec108520 0x7f85ec1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85ec102440 0x7f85ec19a970 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 --2- 172.21.15.99:0/3649234881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ec102e00 0x7f85ec19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 >> 172.21.15.99:0/3649234881 conn(0x7f85ec0793f0 msgr2=0x7f85ec100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:49.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 shutdown_connections 2024-05-14T16:59:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:48.738+0000 7f85f26c0700 1 -- 172.21.15.99:0/3649234881 wait complete. 2024-05-14T16:59:49.303 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 10 2024-05-14T16:59:49.414 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:49 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3649234881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:49.426 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:49 smithi099 podman[71246]: 2024-05-14 16:59:49.106772857 +0000 UTC m=+0.629311929 container init c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0) 2024-05-14T16:59:49.433 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:49 smithi099 podman[71246]: 2024-05-14 16:59:49.110178464 +0000 UTC m=+0.632717546 container start c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:59:49.443 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:49 smithi099 ceph-mds[71278]: -- 172.21.15.99:0/81341480 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558060755a40 con 0x558060728c00 2024-05-14T16:59:49.455 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:49 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g[71274]: starting mds.g at 2024-05-14T16:59:49.464 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:49 smithi099 bash[71246]: c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 2024-05-14T16:59:49.475 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 16:59:49 smithi099 systemd[1]: Started Ceph mds.g for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:49.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:49 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3649234881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:49 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3649234881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:50.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:50.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.590 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:50.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:50.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:50.649 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: Deploying daemon mds.a on smithi099 2024-05-14T16:59:50.672 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.99:6838/913097680,v1:172.21.15.99:6839/913097680] up:boot 2024-05-14T16:59:50.689 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: fsmap 10 up:standby 2024-05-14T16:59:50.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:50 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:59:50.716 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:50.729 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.743 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.755 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.767 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:50.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:50.792 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:50.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: Deploying daemon mds.a on smithi099 2024-05-14T16:59:50.844 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.99:6838/913097680,v1:172.21.15.99:6839/913097680] up:boot 2024-05-14T16:59:50.857 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: fsmap 10 up:standby 2024-05-14T16:59:50.867 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:50 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:59:50.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:50.898 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.908 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:50.940 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:50.951 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:50.963 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:50.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: Deploying daemon mds.a on smithi099 2024-05-14T16:59:50.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.99:6838/913097680,v1:172.21.15.99:6839/913097680] up:boot 2024-05-14T16:59:50.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: fsmap 10 up:standby 2024-05-14T16:59:51.011 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:50 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:59:51.514 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 systemd[1]: Starting Ceph mds.a for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:51.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:51.811 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 podman[71568]: 2024-05-14 16:59:51.446965468 +0000 UTC m=+0.017354707 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:51.818 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 podman[71568]: 2024-05-14 16:59:51.55740014 +0000 UTC m=+0.127789354 container create bf58256064fadc1e27a94963e9e2cb0e9340602a8a0da6eaacde7da5710376f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T16:59:51.826 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 podman[71568]: 2024-05-14 16:59:51.784761335 +0000 UTC m=+0.355150578 container init bf58256064fadc1e27a94963e9e2cb0e9340602a8a0da6eaacde7da5710376f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-14T16:59:51.833 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 podman[71568]: 2024-05-14 16:59:51.787774411 +0000 UTC m=+0.358163634 container start bf58256064fadc1e27a94963e9e2cb0e9340602a8a0da6eaacde7da5710376f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-14T16:59:51.973 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:52.480 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 ceph-mds[71605]: -- 172.21.15.99:0/499936460 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56013d075a40 con 0x56013d048800 2024-05-14T16:59:52.679 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a[71583]: starting mds.a at 2024-05-14T16:59:52.737 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 bash[71568]: bf58256064fadc1e27a94963e9e2cb0e9340602a8a0da6eaacde7da5710376f0 2024-05-14T16:59:52.813 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 16:59:51 smithi099 systemd[1]: Started Ceph mds.a for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:52.827 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:52.838 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:52.851 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:52.857 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:52.864 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:52.871 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:52.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:52 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:52.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:52.934 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:52.941 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:52.947 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:52.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:52.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:52.990 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:52 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:52.998 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:53.009 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:53.024 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:53.030 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:53.035 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:59:53.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:59:53.125 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:52 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:53.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 -- 172.21.15.99:0/2296646206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 msgr2=0x7f76c8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:53.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 --2- 172.21.15.99:0/2296646206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c8103a60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0099c0 tx=0x7f76bc009cd0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 -- 172.21.15.99:0/2296646206 shutdown_connections 2024-05-14T16:59:53.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 --2- 172.21.15.99:0/2296646206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76c8103fa0 0x7f76c8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 --2- 172.21.15.99:0/2296646206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c8109680 0x7f76c8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 --2- 172.21.15.99:0/2296646206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c8103a60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 -- 172.21.15.99:0/2296646206 >> 172.21.15.99:0/2296646206 conn(0x7f76c807a6f0 msgr2=0x7f76c807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:53.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 -- 172.21.15.99:0/2296646206 shutdown_connections 2024-05-14T16:59:53.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.497+0000 7f76d0651700 1 -- 172.21.15.99:0/2296646206 wait complete. 2024-05-14T16:59:53.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 Processor -- start 2024-05-14T16:59:53.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 -- start start 2024-05-14T16:59:53.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c8103fa0 0x7f76c819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76c8109680 0x7f76c8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81137e0 con 0x7f76c81035e0 2024-05-14T16:59:53.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8113660 con 0x7f76c8103fa0 2024-05-14T16:59:53.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76d0651700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8113960 con 0x7f76c8109680 2024-05-14T16:59:53.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76ce3ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76ce3ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c819f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35286/0 (socket says 172.21.15.99:35286) 2024-05-14T16:59:53.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76ce3ed700 1 -- 172.21.15.99:0/4121018376 learned_addr learned my addr 172.21.15.99:0/4121018376 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:53.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.498+0000 7f76cdbec700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c8103fa0 0x7f76c819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76ce3ed700 1 -- 172.21.15.99:0/4121018376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76c8109680 msgr2=0x7f76c8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:53.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76ce3ed700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76c8109680 0x7f76c8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76ce3ed700 1 -- 172.21.15.99:0/4121018376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c8103fa0 msgr2=0x7f76c819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:53.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76ce3ed700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c8103fa0 0x7f76c819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:53.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76ce3ed700 1 -- 172.21.15.99:0/4121018376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c819a260 con 0x7f76c81035e0 2024-05-14T16:59:53.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76ce3ed700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c819f940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76c0007f10 tx=0x7f76c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:53.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c000ddd0 con 0x7f76c81035e0 2024-05-14T16:59:53.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c0004bb0 con 0x7f76c81035e0 2024-05-14T16:59:53.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c00046a0 con 0x7f76c81035e0 2024-05-14T16:59:53.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c819a550 con 0x7f76c81035e0 2024-05-14T16:59:53.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.499+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81a6810 con 0x7f76c81035e0 2024-05-14T16:59:53.832 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:53 smithi150 ceph-mon[36204]: Deploying daemon mds.b on smithi150 2024-05-14T16:59:53.838 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:53 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.99:6840/862670841,v1:172.21.15.99:6841/862670841] up:boot 2024-05-14T16:59:53.846 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:53 smithi150 ceph-mon[36204]: fsmap 11 up:standby 2024-05-14T16:59:53.853 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:53 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:59:53.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:53 smithi193 ceph-mon[35962]: Deploying daemon mds.b on smithi150 2024-05-14T16:59:53.865 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:53 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.99:6840/862670841,v1:172.21.15.99:6841/862670841] up:boot 2024-05-14T16:59:53.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:53 smithi193 ceph-mon[35962]: fsmap 11 up:standby 2024-05-14T16:59:53.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:53 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:59:53.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.500+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c8104cf0 con 0x7f76c81035e0 2024-05-14T16:59:53.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.501+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f76c001c020 con 0x7f76c81035e0 2024-05-14T16:59:53.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.502+0000 7f76c77fe700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f76b00b2900 0x7f76b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:53.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.503+0000 7f76cdbec700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f76b00b2900 0x7f76b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:53.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.503+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f76c00e1fe0 con 0x7f76c81035e0 2024-05-14T16:59:53.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.503+0000 7f76cdbec700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f76b00b2900 0x7f76b00b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009990 tx=0x7f76bc005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:53.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.506+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76c0023030 con 0x7f76c81035e0 2024-05-14T16:59:53.985 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:53 smithi099 ceph-mon[31851]: Deploying daemon mds.b on smithi150 2024-05-14T16:59:53.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:53 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.99:6840/862670841,v1:172.21.15.99:6841/862670841] up:boot 2024-05-14T16:59:53.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:53 smithi099 ceph-mon[31851]: fsmap 11 up:standby 2024-05-14T16:59:54.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:53 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:59:54.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.752+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76c802e560 con 0x7f76c81035e0 2024-05-14T16:59:54.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.753+0000 7f76c77fe700 1 -- 172.21.15.99:0/4121018376 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f76c0010070 con 0x7f76c81035e0 2024-05-14T16:59:54.031 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:54.039 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-14T16:59:54.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f76b00b2900 msgr2=0x7f76b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f76b00b2900 0x7f76b00b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009990 tx=0x7f76bc005c40 comp rx=0 tx=0).stop 2024-05-14T16:59:54.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 msgr2=0x7f76c819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:54.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c819f940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76c0007f10 tx=0x7f76c000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 shutdown_connections 2024-05-14T16:59:54.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f76b00b2900 0x7f76b00b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76c8109680 0x7f76c8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81035e0 0x7f76c819f940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 --2- 172.21.15.99:0/4121018376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c8103fa0 0x7f76c819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:54.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.755+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 >> 172.21.15.99:0/4121018376 conn(0x7f76c807a6f0 msgr2=0x7f76c8102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:54.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.756+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 shutdown_connections 2024-05-14T16:59:54.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:53.756+0000 7f76d0651700 1 -- 172.21.15.99:0/4121018376 wait complete. 2024-05-14T16:59:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 12 2024-05-14T16:59:54.220 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 systemd[1]: Starting Ceph mds.b for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T16:59:54.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:54 smithi099 ceph-mon[31851]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:54.402 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:54 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4121018376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:54.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:54 smithi150 ceph-mon[36204]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:54.546 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:54 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4121018376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:54.553 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 podman[66590]: 2024-05-14 16:59:54.296787147 +0000 UTC m=+0.106368709 container create d5eb5f5c43f62ec987a2b0de7714fd5d366bb35c71e5ad16fa7fe48e552cec85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:59:54.560 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 podman[66590]: 2024-05-14 16:59:54.207497569 +0000 UTC m=+0.017079134 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:59:54.566 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 podman[66590]: 2024-05-14 16:59:54.507644042 +0000 UTC m=+0.317225596 container init d5eb5f5c43f62ec987a2b0de7714fd5d366bb35c71e5ad16fa7fe48e552cec85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:59:54.573 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 podman[66590]: 2024-05-14 16:59:54.51017825 +0000 UTC m=+0.319759821 container start d5eb5f5c43f62ec987a2b0de7714fd5d366bb35c71e5ad16fa7fe48e552cec85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:59:54.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:54 smithi193 ceph-mon[35962]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:54.586 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:54 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4121018376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:54.953 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 ceph-mds[66609]: -- 172.21.15.150:0/2785175497 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf9c69ba40 con 0x55bf9c66e800 2024-05-14T16:59:54.962 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b[66605]: starting mds.b at 2024-05-14T16:59:54.969 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 bash[66590]: d5eb5f5c43f62ec987a2b0de7714fd5d366bb35c71e5ad16fa7fe48e552cec85 2024-05-14T16:59:54.976 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 16:59:54 smithi150 systemd[1]: Started Ceph mds.b for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T16:59:55.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:55.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:55.968 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:55.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:55.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:55.987 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:55.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:56.027 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] up:boot 2024-05-14T16:59:56.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: fsmap 12 up:standby 2024-05-14T16:59:56.041 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:55 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:59:56.048 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.060 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.085 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.155 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:56.176 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] up:boot 2024-05-14T16:59:56.185 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: fsmap 12 up:standby 2024-05-14T16:59:56.193 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:55 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:59:56.200 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:56.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.213 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:56.248 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:56.254 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] up:boot 2024-05-14T16:59:56.262 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: fsmap 12 up:standby 2024-05-14T16:59:56.268 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:55 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:59:56.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:56.678 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:57.688 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.696 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:57.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.717 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.732 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.738 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:57.803 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:57.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:57 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:57.856 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.868 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:57.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.902 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.952 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.958 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.965 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:57.971 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:57.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:57.985 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:57 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:58.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.017 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:58.052 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.069 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.076 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.082 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.088 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:58.098 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:58.106 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:57 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:58.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.741 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.748 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.755 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:58 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.768 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.781 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.791 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.798 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:58 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.854 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.862 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.869 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.882 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:58 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:58.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 -- 172.21.15.99:0/3664544473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102e00 msgr2=0x7f830c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:58.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 --2- 172.21.15.99:0/3664544473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102e00 0x7f830c10f2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f830400b0a0 tx=0x7f830400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 -- 172.21.15.99:0/3664544473 shutdown_connections 2024-05-14T16:59:59.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 --2- 172.21.15.99:0/3664544473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102e00 0x7f830c10f2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 --2- 172.21.15.99:0/3664544473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 0x7f830c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 --2- 172.21.15.99:0/3664544473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c102440 0x7f830c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 -- 172.21.15.99:0/3664544473 >> 172.21.15.99:0/3664544473 conn(0x7f830c0793f0 msgr2=0x7f830c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:59.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 -- 172.21.15.99:0/3664544473 shutdown_connections 2024-05-14T16:59:59.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.809+0000 7f8311872700 1 -- 172.21.15.99:0/3664544473 wait complete. 2024-05-14T16:59:59.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 Processor -- start 2024-05-14T16:59:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 -- start start 2024-05-14T16:59:59.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 0x7f830c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:59.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c102e00 0x7f830c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:59.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 0x7f830c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1125d0 con 0x7f830c102440 2024-05-14T16:59:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f830c112450 con 0x7f830c108520 2024-05-14T16:59:59.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f8311872700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f830c112750 con 0x7f830c102e00 2024-05-14T16:59:59.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f830affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 0x7f830c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:59.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f830affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 0x7f830c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59086/0 (socket says 172.21.15.99:59086) 2024-05-14T16:59:59.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.810+0000 7f830affd700 1 -- 172.21.15.99:0/374415200 learned_addr learned my addr 172.21.15.99:0/374415200 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T16:59:59.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830a7fc700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c102e00 0x7f830c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:59.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830b7fe700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 0x7f830c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:59.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830affd700 1 -- 172.21.15.99:0/374415200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c102e00 msgr2=0x7f830c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:59.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830affd700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c102e00 0x7f830c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830affd700 1 -- 172.21.15.99:0/374415200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 msgr2=0x7f830c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830affd700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 0x7f830c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830affd700 1 -- 172.21.15.99:0/374415200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c19a1b0 con 0x7f830c102440 2024-05-14T16:59:59.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830b7fe700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 0x7f830c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:59.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f830affd700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 0x7f830c19f840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00c920 tx=0x7f82fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:59.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc007b00 con 0x7f830c102440 2024-05-14T16:59:59.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c19a4a0 con 0x7f830c102440 2024-05-14T16:59:59.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82fc004ce0 con 0x7f830c102440 2024-05-14T16:59:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc005500 con 0x7f830c102440 2024-05-14T16:59:59.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a6710 con 0x7f830c102440 2024-05-14T16:59:59.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.812+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c06a230 con 0x7f830c102440 2024-05-14T16:59:59.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.815+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f82fc007ca0 con 0x7f830c102440 2024-05-14T16:59:59.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.815+0000 7f8310870700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f82f40b2880 0x7f82f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:59.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.815+0000 7f830a7fc700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f82f40b2880 0x7f82f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:59.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.815+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f82fc0e1c80 con 0x7f830c102440 2024-05-14T16:59:59.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.815+0000 7f830a7fc700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f82f40b2880 0x7f82f40b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f830c079ba0 tx=0x7f8300009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:59.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:58.822+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82fc014030 con 0x7f830c102440 2024-05-14T16:59:59.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.099+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f830c19af30 con 0x7f830c102440 2024-05-14T16:59:59.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.100+0000 7f8310870700 1 -- 172.21.15.99:0/374415200 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f82fc0a9e40 con 0x7f830c102440 2024-05-14T16:59:59.498 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T16:59:59.505 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24413,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-14T16:59:59.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.102+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f82f40b2880 msgr2=0x7f82f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:59.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.102+0000 7f8311872700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f82f40b2880 0x7f82f40b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f830c079ba0 tx=0x7f8300009260 comp rx=0 tx=0).stop 2024-05-14T16:59:59.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.102+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 msgr2=0x7f830c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:59.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.102+0000 7f8311872700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 0x7f830c19f840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00c920 tx=0x7f82fc00cc30 comp rx=0 tx=0).stop 2024-05-14T16:59:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 shutdown_connections 2024-05-14T16:59:59.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f82f40b2880 0x7f82f40b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c108520 0x7f830c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830c102440 0x7f830c19f840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 --2- 172.21.15.99:0/374415200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c102e00 0x7f830c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:59.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 >> 172.21.15.99:0/374415200 conn(0x7f830c0793f0 msgr2=0x7f830c06b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:59.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 shutdown_connections 2024-05-14T16:59:59.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T16:59:59.103+0000 7f8311872700 1 -- 172.21.15.99:0/374415200 wait complete. 2024-05-14T16:59:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 13 2024-05-14T16:59:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:59 smithi150 ceph-mon[36204]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:59.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:59 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/374415200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:59.716 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:59.724 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 16:59:59 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:59.745 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:59 smithi099 ceph-mon[31851]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:59.773 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:59 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/374415200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:59.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:59.789 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 16:59:59 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:59.795 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:59 smithi193 ceph-mon[35962]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:59:59.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:59 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/374415200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:59.811 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:59.827 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 16:59:59 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T16:59:59.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-14T17:00:00.109 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:00.885 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:00 smithi099 ceph-mon[31851]: overall HEALTH_OK 2024-05-14T17:00:00.899 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:00.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:00.912 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:00:01.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:00 smithi193 ceph-mon[35962]: overall HEALTH_OK 2024-05-14T17:00:01.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:01.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:01.064 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:00:01.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:00 smithi150 ceph-mon[36204]: overall HEALTH_OK 2024-05-14T17:00:01.314 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:01.335 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:01.355 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:00:01.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 -- 172.21.15.99:0/339686059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90075770 msgr2=0x7fef90075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:01.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 --2- 172.21.15.99:0/339686059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90075770 0x7fef90075b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009990 tx=0x7fef8c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 -- 172.21.15.99:0/339686059 shutdown_connections 2024-05-14T17:00:01.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 --2- 172.21.15.99:0/339686059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef9010ff40 0x7fef90112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 --2- 172.21.15.99:0/339686059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90075770 0x7fef90075b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 --2- 172.21.15.99:0/339686059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef90076090 0x7fef9010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 -- 172.21.15.99:0/339686059 >> 172.21.15.99:0/339686059 conn(0x7fef9007a6a0 msgr2=0x7fef9007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 -- 172.21.15.99:0/339686059 shutdown_connections 2024-05-14T17:00:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.658+0000 7fef97a07700 1 -- 172.21.15.99:0/339686059 wait complete. 2024-05-14T17:00:02.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 Processor -- start 2024-05-14T17:00:02.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 -- start start 2024-05-14T17:00:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90075770 0x7fef9019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 0x7fef9019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef9010ff40 0x7fef901a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef901158e0 con 0x7fef90076090 2024-05-14T17:00:02.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef90115760 con 0x7fef90075770 2024-05-14T17:00:02.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef97a07700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef90115a60 con 0x7fef9010ff40 2024-05-14T17:00:02.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 0x7fef9019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:02.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 0x7fef9019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59104/0 (socket says 172.21.15.99:59104) 2024-05-14T17:00:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 -- 172.21.15.99:0/1169617143 learned_addr learned my addr 172.21.15.99:0/1169617143 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef95fa4700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef9010ff40 0x7fef901a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:03.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef957a3700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90075770 0x7fef9019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:03.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 -- 172.21.15.99:0/1169617143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef9010ff40 msgr2=0x7fef901a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef9010ff40 0x7fef901a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 -- 172.21.15.99:0/1169617143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90075770 msgr2=0x7fef9019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90075770 0x7fef9019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef94fa2700 1 -- 172.21.15.99:0/1169617143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef901a0dd0 con 0x7fef90076090 2024-05-14T17:00:03.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef95fa4700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef9010ff40 0x7fef901a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:03.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.659+0000 7fef957a3700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90075770 0x7fef9019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:03.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.660+0000 7fef94fa2700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 0x7fef9019c350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef8000d980 tx=0x7fef8000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:03.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.660+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80017070 con 0x7fef90076090 2024-05-14T17:00:03.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.660+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef80004830 con 0x7fef90076090 2024-05-14T17:00:03.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.660+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef901a10c0 con 0x7fef90076090 2024-05-14T17:00:03.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.660+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80005440 con 0x7fef90076090 2024-05-14T17:00:03.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.660+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef901a8940 con 0x7fef90076090 2024-05-14T17:00:03.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.661+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9006a230 con 0x7fef90076090 2024-05-14T17:00:03.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.662+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fef800055e0 con 0x7fef90076090 2024-05-14T17:00:03.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.663+0000 7fef867fc700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef7c0b2880 0x7fef7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.663+0000 7fef957a3700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef7c0b2880 0x7fef7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:03.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.663+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fef800e2190 con 0x7fef90076090 2024-05-14T17:00:03.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.664+0000 7fef957a3700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef7c0b2880 0x7fef7c0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b5c0 tx=0x7fef8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:03.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.667+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef80020020 con 0x7fef90076090 2024-05-14T17:00:03.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:02 smithi150 ceph-mon[36204]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:03.579 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:02 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1169617143' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T17:00:03.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:02 smithi193 ceph-mon[35962]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:03.593 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:02 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1169617143' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T17:00:03.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.871+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fef9002e560 con 0x7fef90076090 2024-05-14T17:00:03.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.871+0000 7fef867fc700 1 -- 172.21.15.99:0/1169617143 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7fef800aa2b0 con 0x7fef90076090 2024-05-14T17:00:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:03.646 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-05-14T17:00:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:00:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-05-14T17:00:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-05-14T17:00:03.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:00:03.716 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-05-14T17:00:03.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef7c0b2880 msgr2=0x7fef7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef7c0b2880 0x7fef7c0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00b5c0 tx=0x7fef8c005fb0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 msgr2=0x7fef9019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 0x7fef9019c350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef8000d980 tx=0x7fef8000dd40 comp rx=0 tx=0).stop 2024-05-14T17:00:03.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 shutdown_connections 2024-05-14T17:00:03.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef7c0b2880 0x7fef7c0b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef9010ff40 0x7fef901a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef90075770 0x7fef9019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 --2- 172.21.15.99:0/1169617143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef90076090 0x7fef9019c350 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 >> 172.21.15.99:0/1169617143 conn(0x7fef9007a6a0 msgr2=0x7fef901007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:03.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 shutdown_connections 2024-05-14T17:00:03.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:01.874+0000 7fef97a07700 1 -- 172.21.15.99:0/1169617143 wait complete. 2024-05-14T17:00:03.819 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:02 smithi099 ceph-mon[31851]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:03.825 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:02 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1169617143' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T17:00:03.832 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T17:00:03.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T17:00:04.138 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:04.260 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:04 smithi099 ceph-mon[31851]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:04.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:04 smithi193 ceph-mon[35962]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:04 smithi150 ceph-mon[36204]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:05.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 -- 172.21.15.99:0/3382081701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a41028c0 msgr2=0x7f06a4110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:05.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 --2- 172.21.15.99:0/3382081701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a41028c0 0x7f06a4110330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f069c00b0a0 tx=0x7f069c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 -- 172.21.15.99:0/3382081701 shutdown_connections 2024-05-14T17:00:05.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 --2- 172.21.15.99:0/3382081701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a41028c0 0x7f06a4110330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 --2- 172.21.15.99:0/3382081701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4108620 0x7f06a4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 --2- 172.21.15.99:0/3382081701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a4101f00 0x7f06a4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 -- 172.21.15.99:0/3382081701 >> 172.21.15.99:0/3382081701 conn(0x7f06a4077780 msgr2=0x7f06a4077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 -- 172.21.15.99:0/3382081701 shutdown_connections 2024-05-14T17:00:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 -- 172.21.15.99:0/3382081701 wait complete. 2024-05-14T17:00:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 Processor -- start 2024-05-14T17:00:05.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.775+0000 7f06abe33700 1 -- start start 2024-05-14T17:00:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4101f00 0x7f06a4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a41028c0 0x7f06a419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:05.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 0x7f06a419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4113830 con 0x7f06a4108620 2024-05-14T17:00:06.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41136b0 con 0x7f06a41028c0 2024-05-14T17:00:06.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41139b0 con 0x7f06a4101f00 2024-05-14T17:00:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 0x7f06a419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 0x7f06a419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56826/0 (socket says 172.21.15.99:56826) 2024-05-14T17:00:06.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 -- 172.21.15.99:0/182093247 learned_addr learned my addr 172.21.15.99:0/182093247 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06a9bcf700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4101f00 0x7f06a4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06a93ce700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a41028c0 0x7f06a419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 -- 172.21.15.99:0/182093247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4101f00 msgr2=0x7f06a4199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4101f00 0x7f06a4199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 -- 172.21.15.99:0/182093247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a41028c0 msgr2=0x7f06a419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a41028c0 0x7f06a419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 -- 172.21.15.99:0/182093247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a419ed30 con 0x7f06a4108620 2024-05-14T17:00:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06a9bcf700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4101f00 0x7f06a4199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06aa3d0700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 0x7f06a419e5f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f069c0062a0 tx=0x7f069c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:06.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c021070 con 0x7f06a4108620 2024-05-14T17:00:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c007d00 con 0x7f06a4108620 2024-05-14T17:00:06.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c01d450 con 0x7f06a4108620 2024-05-14T17:00:06.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a419efc0 con 0x7f06a4108620 2024-05-14T17:00:06.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.776+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41a68a0 con 0x7f06a4108620 2024-05-14T17:00:06.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.777+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a406a230 con 0x7f06a4108620 2024-05-14T17:00:06.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.779+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f069c004000 con 0x7f06a4108620 2024-05-14T17:00:06.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.779+0000 7f069affd700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f06900b2900 0x7f06900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.780+0000 7f06a9bcf700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f06900b2900 0x7f06900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.780+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f069c0e2f20 con 0x7f06a4108620 2024-05-14T17:00:06.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.780+0000 7f06a9bcf700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f06900b2900 0x7f06900b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f06a4078ce0 tx=0x7f06a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:06.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.783+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f069c0ab170 con 0x7f06a4108620 2024-05-14T17:00:06.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:05.985+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f06a402e560 con 0x7f06a4108620 2024-05-14T17:00:06.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.016+0000 7f069affd700 1 -- 172.21.15.99:0/182093247 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f069c0aaa70 con 0x7f06a4108620 2024-05-14T17:00:06.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f06900b2900 msgr2=0x7f06900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f06900b2900 0x7f06900b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f06a4078ce0 tx=0x7f06a0008040 comp rx=0 tx=0).stop 2024-05-14T17:00:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 msgr2=0x7f06a419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 0x7f06a419e5f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f069c0062a0 tx=0x7f069c004ed0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 shutdown_connections 2024-05-14T17:00:06.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f06900b2900 0x7f06900b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a4108620 0x7f06a419e5f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4101f00 0x7f06a4199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 --2- 172.21.15.99:0/182093247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a41028c0 0x7f06a419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:06.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 >> 172.21.15.99:0/182093247 conn(0x7f06a4077780 msgr2=0x7f06a4100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:06.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.019+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 shutdown_connections 2024-05-14T17:00:06.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:06.020+0000 7f06abe33700 1 -- 172.21.15.99:0/182093247 wait complete. 2024-05-14T17:00:06.413 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-05-14T17:00:06.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:06 smithi193 ceph-mon[35962]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:06.427 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:06 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/182093247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T17:00:06.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:06 smithi099 ceph-mon[31851]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:06.440 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:06 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/182093247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T17:00:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:06 smithi150 ceph-mon[36204]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:06.459 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:06 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/182093247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T17:00:06.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T17:00:06.886 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:07 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/182093247' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T17:00:07.271 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:07 smithi099 ceph-mon[31851]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T17:00:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:07 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/182093247' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T17:00:07.299 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:07 smithi193 ceph-mon[35962]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T17:00:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:07 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/182093247' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T17:00:07.504 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:07 smithi150 ceph-mon[36204]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T17:00:08.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:08 smithi099 ceph-mon[31851]: pgmap v276: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:08.272 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:08 smithi099 ceph-mon[31851]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T17:00:08.279 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:08 smithi099 ceph-mon[31851]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T17:00:08.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:08 smithi193 ceph-mon[35962]: pgmap v276: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:08.299 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:08 smithi193 ceph-mon[35962]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T17:00:08.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:08 smithi193 ceph-mon[35962]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T17:00:08.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:08 smithi150 ceph-mon[36204]: pgmap v276: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:08.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:08 smithi150 ceph-mon[36204]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T17:00:08.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:08 smithi150 ceph-mon[36204]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T17:00:08.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.484+0000 7f0f54e92700 1 -- 172.21.15.99:0/2861472653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501037f0 msgr2=0x7f0f50103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:08.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.484+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2861472653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501037f0 0x7f0f50103bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f380099c0 tx=0x7f0f38009cd0 comp rx=0 tx=0).stop 2024-05-14T17:00:08.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 -- 172.21.15.99:0/2861472653 shutdown_connections 2024-05-14T17:00:08.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2861472653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f501088f0 0x7f0f50110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:08.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2861472653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501037f0 0x7f0f50103bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:08.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2861472653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f501041a0 0x7f0f501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:08.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 -- 172.21.15.99:0/2861472653 >> 172.21.15.99:0/2861472653 conn(0x7f0f5007a6a0 msgr2=0x7f0f5007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:08.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 -- 172.21.15.99:0/2861472653 shutdown_connections 2024-05-14T17:00:08.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 -- 172.21.15.99:0/2861472653 wait complete. 2024-05-14T17:00:08.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 Processor -- start 2024-05-14T17:00:08.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.485+0000 7f0f54e92700 1 -- start start 2024-05-14T17:00:08.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f54e92700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f501037f0 0x7f0f50075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:08.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f54e92700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f501041a0 0x7f0f50075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f54e92700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 0x7f0f500762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:08.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f54e92700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50113800 con 0x7f0f501088f0 2024-05-14T17:00:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f54e92700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50113680 con 0x7f0f501041a0 2024-05-14T17:00:08.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f54e92700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50113980 con 0x7f0f501037f0 2024-05-14T17:00:08.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 0x7f0f500762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:08.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 0x7f0f500762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56852/0 (socket says 172.21.15.99:56852) 2024-05-14T17:00:08.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 -- 172.21.15.99:0/2022088519 learned_addr learned my addr 172.21.15.99:0/2022088519 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:08.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4e59c700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f501037f0 0x7f0f50075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:08.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 -- 172.21.15.99:0/2022088519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f501037f0 msgr2=0x7f0f50075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4dd9b700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f501041a0 0x7f0f50075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:08.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f501037f0 0x7f0f50075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:08.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 -- 172.21.15.99:0/2022088519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f501041a0 msgr2=0x7f0f50075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:08.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f501041a0 0x7f0f50075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:08.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 -- 172.21.15.99:0/2022088519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f501a8240 con 0x7f0f501088f0 2024-05-14T17:00:08.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f4ed9d700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 0x7f0f500762c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800b970 tx=0x7f0f4800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:08.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48019070 con 0x7f0f501088f0 2024-05-14T17:00:08.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f48014440 con 0x7f0f501088f0 2024-05-14T17:00:08.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.486+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800ce60 con 0x7f0f501088f0 2024-05-14T17:00:09.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.487+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f501a8530 con 0x7f0f501088f0 2024-05-14T17:00:09.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.487+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f501a8a20 con 0x7f0f501088f0 2024-05-14T17:00:09.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.488+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5010bb30 con 0x7f0f501088f0 2024-05-14T17:00:09.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.489+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0f4801c070 con 0x7f0f501088f0 2024-05-14T17:00:09.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.490+0000 7f0f477fe700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0f3c0b2880 0x7f0f3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:09.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.490+0000 7f0f4e59c700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0f3c0b2880 0x7f0f3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.490+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f0f480e1800 con 0x7f0f501088f0 2024-05-14T17:00:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.490+0000 7f0f4e59c700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0f3c0b2880 0x7f0f3c0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009f50 tx=0x7f0f3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:09.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.493+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f480a9850 con 0x7f0f501088f0 2024-05-14T17:00:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:08.696+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0f5010b510 con 0x7f0f501088f0 2024-05-14T17:00:09.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.037+0000 7f0f477fe700 1 -- 172.21.15.99:0/2022088519 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f0f480a9150 con 0x7f0f501088f0 2024-05-14T17:00:09.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0f3c0b2880 msgr2=0x7f0f3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:09.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0f3c0b2880 0x7f0f3c0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009f50 tx=0x7f0f3801a040 comp rx=0 tx=0).stop 2024-05-14T17:00:09.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 msgr2=0x7f0f500762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:09.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 0x7f0f500762c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800b970 tx=0x7f0f4800bd30 comp rx=0 tx=0).stop 2024-05-14T17:00:09.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 shutdown_connections 2024-05-14T17:00:09.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f0f3c0b2880 0x7f0f3c0b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:09.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f501088f0 0x7f0f500762c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:09.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f501037f0 0x7f0f50075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:09.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 --2- 172.21.15.99:0/2022088519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f501041a0 0x7f0f50075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 >> 172.21.15.99:0/2022088519 conn(0x7f0f5007a6a0 msgr2=0x7f0f501018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:09.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 shutdown_connections 2024-05-14T17:00:09.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:09.041+0000 7f0f54e92700 1 -- 172.21.15.99:0/2022088519 wait complete. 2024-05-14T17:00:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-05-14T17:00:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:09 smithi193 ceph-mon[35962]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T17:00:09.298 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:09 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2022088519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T17:00:09.334 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:09 smithi099 ceph-mon[31851]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T17:00:09.341 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:09 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2022088519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T17:00:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:09 smithi150 ceph-mon[36204]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T17:00:09.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:09 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2022088519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T17:00:09.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T17:00:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:10.299 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:10 smithi099 ceph-mon[31851]: pgmap v279: 33 pgs: 6 creating+peering, 2 active+clean, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:10.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:10 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2022088519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T17:00:10.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:10 smithi099 ceph-mon[31851]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T17:00:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:10 smithi150 ceph-mon[36204]: pgmap v279: 33 pgs: 6 creating+peering, 2 active+clean, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:10.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:10 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2022088519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T17:00:10.487 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:10 smithi150 ceph-mon[36204]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T17:00:10.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:10 smithi193 ceph-mon[35962]: pgmap v279: 33 pgs: 6 creating+peering, 2 active+clean, 25 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:10.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:10 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2022088519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T17:00:10.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:10 smithi193 ceph-mon[35962]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T17:00:11.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:11 smithi099 ceph-mon[31851]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T17:00:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:11 smithi150 ceph-mon[36204]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T17:00:11.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:11 smithi193 ceph-mon[35962]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T17:00:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 -- 172.21.15.99:0/869686826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 msgr2=0x7fe21c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 --2- 172.21.15.99:0/869686826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c10f310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe21400b0a0 tx=0x7fe21400b3b0 comp rx=0 tx=0).stop 2024-05-14T17:00:11.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 -- 172.21.15.99:0/869686826 shutdown_connections 2024-05-14T17:00:11.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 --2- 172.21.15.99:0/869686826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c10f310 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:11.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 --2- 172.21.15.99:0/869686826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe21c1084d0 0x7fe21c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:11.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 --2- 172.21.15.99:0/869686826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe21c1024d0 0x7fe21c102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:11.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 -- 172.21.15.99:0/869686826 >> 172.21.15.99:0/869686826 conn(0x7fe21c078490 msgr2=0x7fe21c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:11.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 -- 172.21.15.99:0/869686826 shutdown_connections 2024-05-14T17:00:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.594+0000 7fe2243a8700 1 -- 172.21.15.99:0/869686826 wait complete. 2024-05-14T17:00:11.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 Processor -- start 2024-05-14T17:00:11.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 -- start start 2024-05-14T17:00:11.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe21c1024d0 0x7fe21c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:11.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:11.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe21c1084d0 0x7fe21c1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c112680 con 0x7fe21c102e90 2024-05-14T17:00:11.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c112500 con 0x7fe21c1084d0 2024-05-14T17:00:11.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe2243a8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c112800 con 0x7fe21c1024d0 2024-05-14T17:00:11.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe221943700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:11.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe221943700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56866/0 (socket says 172.21.15.99:56866) 2024-05-14T17:00:11.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe222144700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe21c1024d0 0x7fe21c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:11.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.595+0000 7fe221943700 1 -- 172.21.15.99:0/2365860904 learned_addr learned my addr 172.21.15.99:0/2365860904 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:11.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe222945700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe21c1084d0 0x7fe21c1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:11.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe221943700 1 -- 172.21.15.99:0/2365860904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe21c1024d0 msgr2=0x7fe21c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:11.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe221943700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe21c1024d0 0x7fe21c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe221943700 1 -- 172.21.15.99:0/2365860904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe21c1084d0 msgr2=0x7fe21c1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:11.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe221943700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe21c1084d0 0x7fe21c1a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:11.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe221943700 1 -- 172.21.15.99:0/2365860904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c1a6080 con 0x7fe21c102e90 2024-05-14T17:00:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe221943700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c19b000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe21800b530 tx=0x7fe21800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:11.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218015020 con 0x7fe21c102e90 2024-05-14T17:00:11.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe218011ce0 con 0x7fe21c102e90 2024-05-14T17:00:12.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21c1a6250 con 0x7fe21c102e90 2024-05-14T17:00:12.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218010e40 con 0x7fe21c102e90 2024-05-14T17:00:12.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.596+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21c1a67b0 con 0x7fe21c102e90 2024-05-14T17:00:12.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.597+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21c06a230 con 0x7fe21c102e90 2024-05-14T17:00:12.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.599+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe218018020 con 0x7fe21c102e90 2024-05-14T17:00:12.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.600+0000 7fe20f7fe700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe2080b2900 0x7fe2080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:12.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.600+0000 7fe222144700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe2080b2900 0x7fe2080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:12.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.600+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fe2180e1dd0 con 0x7fe21c102e90 2024-05-14T17:00:12.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.601+0000 7fe222144700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe2080b2900 0x7fe2080b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe210004630 tx=0x7fe21000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:12.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.604+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe21801f020 con 0x7fe21c102e90 2024-05-14T17:00:12.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:11.850+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe21c02e560 con 0x7fe21c102e90 2024-05-14T17:00:12.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.052+0000 7fe20f7fe700 1 -- 172.21.15.99:0/2365860904 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe2180a9ca0 con 0x7fe21c102e90 2024-05-14T17:00:12.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe2080b2900 msgr2=0x7fe2080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:12.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe2080b2900 0x7fe2080b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe210004630 tx=0x7fe21000b3c0 comp rx=0 tx=0).stop 2024-05-14T17:00:12.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 msgr2=0x7fe21c19b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:12.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c19b000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe21800b530 tx=0x7fe21800b840 comp rx=0 tx=0).stop 2024-05-14T17:00:12.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 shutdown_connections 2024-05-14T17:00:12.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe2080b2900 0x7fe2080b4dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:12.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe21c1084d0 0x7fe21c1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:12.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe21c1024d0 0x7fe21c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:12.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 --2- 172.21.15.99:0/2365860904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c102e90 0x7fe21c19b000 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:12.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 >> 172.21.15.99:0/2365860904 conn(0x7fe21c078490 msgr2=0x7fe21c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:12.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 shutdown_connections 2024-05-14T17:00:12.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:12.055+0000 7fe2243a8700 1 -- 172.21.15.99:0/2365860904 wait complete. 2024-05-14T17:00:12.226 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2024-05-14T17:00:12.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:12 smithi099 ceph-mon[31851]: pgmap v282: 97 pgs: 25 creating+peering, 24 active+clean, 48 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:12.270 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:12 smithi099 ceph-mon[31851]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T17:00:12.277 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:12 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2365860904' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T17:00:12.282 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:12 smithi099 ceph-mon[31851]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-14T17:00:12.354 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:12 smithi150 ceph-mon[36204]: pgmap v282: 97 pgs: 25 creating+peering, 24 active+clean, 48 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:12.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:12 smithi150 ceph-mon[36204]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T17:00:12.378 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:12 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2365860904' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T17:00:12.388 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:12 smithi150 ceph-mon[36204]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-14T17:00:12.444 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:12 smithi193 ceph-mon[35962]: pgmap v282: 97 pgs: 25 creating+peering, 24 active+clean, 48 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:12.455 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:12 smithi193 ceph-mon[35962]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T17:00:12.461 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:12 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2365860904' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T17:00:12.467 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:12 smithi193 ceph-mon[35962]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-14T17:00:12.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:00:12.938 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:13.367 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:13 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2365860904' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T17:00:13.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:13 smithi150 ceph-mon[36204]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T17:00:13.388 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:13 smithi150 ceph-mon[36204]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-14T17:00:13.397 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:13 smithi150 ceph-mon[36204]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-14T17:00:13.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:13 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2365860904' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T17:00:13.412 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:13 smithi099 ceph-mon[31851]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T17:00:13.418 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:13 smithi099 ceph-mon[31851]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-14T17:00:13.427 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:13 smithi099 ceph-mon[31851]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-14T17:00:13.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:13 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2365860904' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T17:00:13.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:13 smithi193 ceph-mon[35962]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T17:00:13.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:13 smithi193 ceph-mon[35962]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-14T17:00:13.769 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:13 smithi193 ceph-mon[35962]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-14T17:00:14.324 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:14 smithi099 ceph-mon[31851]: pgmap v285: 97 pgs: 19 creating+peering, 57 active+clean, 21 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:14.334 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:14 smithi099 ceph-mon[31851]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T17:00:14.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:14 smithi099 ceph-mon[31851]: Cluster is now healthy 2024-05-14T17:00:14.360 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:14 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] up:active 2024-05-14T17:00:14.374 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:14 smithi099 ceph-mon[31851]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:14.408 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:14 smithi099 ceph-mon[31851]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:14.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:14 smithi150 ceph-mon[36204]: pgmap v285: 97 pgs: 19 creating+peering, 57 active+clean, 21 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:14.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:14 smithi150 ceph-mon[36204]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T17:00:14.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:14 smithi150 ceph-mon[36204]: Cluster is now healthy 2024-05-14T17:00:14.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:14 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] up:active 2024-05-14T17:00:14.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:14 smithi150 ceph-mon[36204]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:14.512 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:14 smithi150 ceph-mon[36204]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:14.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 -- 172.21.15.99:0/3468260269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1024d0 msgr2=0x7f1dfc102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:14.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 --2- 172.21.15.99:0/3468260269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1024d0 0x7f1dfc102950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009990 tx=0x7f1dec009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 -- 172.21.15.99:0/3468260269 shutdown_connections 2024-05-14T17:00:14.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 --2- 172.21.15.99:0/3468260269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dfc102e90 0x7f1dfc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 --2- 172.21.15.99:0/3468260269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 --2- 172.21.15.99:0/3468260269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1024d0 0x7f1dfc102950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 -- 172.21.15.99:0/3468260269 >> 172.21.15.99:0/3468260269 conn(0x7f1dfc078490 msgr2=0x7f1dfc0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:14.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 -- 172.21.15.99:0/3468260269 shutdown_connections 2024-05-14T17:00:14.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.513+0000 7f1e038e5700 1 -- 172.21.15.99:0/3468260269 wait complete. 2024-05-14T17:00:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 Processor -- start 2024-05-14T17:00:14.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 -- start start 2024-05-14T17:00:14.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dfc1024d0 0x7f1dfc1040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc102e90 0x7f1dfc1045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1125e0 con 0x7f1dfc1084d0 2024-05-14T17:00:14.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112460 con 0x7f1dfc102e90 2024-05-14T17:00:14.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e038e5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112760 con 0x7f1dfc1024d0 2024-05-14T17:00:14.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e01e82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e01e82700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39494/0 (socket says 172.21.15.99:39494) 2024-05-14T17:00:14.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e01e82700 1 -- 172.21.15.99:0/1556368065 learned_addr learned my addr 172.21.15.99:0/1556368065 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:15.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e01681700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dfc1024d0 0x7f1dfc1040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.514+0000 7f1e00e80700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc102e90 0x7f1dfc1045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e01e82700 1 -- 172.21.15.99:0/1556368065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dfc1024d0 msgr2=0x7f1dfc1040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e01e82700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dfc1024d0 0x7f1dfc1040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e01e82700 1 -- 172.21.15.99:0/1556368065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc102e90 msgr2=0x7f1dfc1045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e01e82700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc102e90 0x7f1dfc1045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e01e82700 1 -- 172.21.15.99:0/1556368065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc1a80e0 con 0x7f1dfc1084d0 2024-05-14T17:00:15.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e00e80700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc102e90 0x7f1dfc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:15.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e01e82700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc104b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007d70 tx=0x7f1df400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:15.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4016020 con 0x7f1dfc1084d0 2024-05-14T17:00:15.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc1a83d0 con 0x7f1dfc1084d0 2024-05-14T17:00:15.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df400ac90 con 0x7f1dfc1084d0 2024-05-14T17:00:15.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df40115e0 con 0x7f1dfc1084d0 2024-05-14T17:00:15.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.515+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1a88e0 con 0x7f1dfc1084d0 2024-05-14T17:00:15.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.516+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc06a230 con 0x7f1dfc1084d0 2024-05-14T17:00:15.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.518+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1df4019020 con 0x7f1dfc1084d0 2024-05-14T17:00:15.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.519+0000 7f1df27fc700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1de80b2880 0x7f1de80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.519+0000 7f1e01681700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1de80b2880 0x7f1de80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.519+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1df40e24e0 con 0x7f1dfc1084d0 2024-05-14T17:00:15.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.519+0000 7f1e01681700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1de80b2880 0x7f1de80b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007b60 tx=0x7f1df80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:15.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.522+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1df4020020 con 0x7f1dfc1084d0 2024-05-14T17:00:15.288 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:14 smithi193 ceph-mon[35962]: pgmap v285: 97 pgs: 19 creating+peering, 57 active+clean, 21 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:00:15.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:14 smithi193 ceph-mon[35962]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T17:00:15.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:14 smithi193 ceph-mon[35962]: Cluster is now healthy 2024-05-14T17:00:15.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:14 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] up:active 2024-05-14T17:00:15.344 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:14 smithi193 ceph-mon[35962]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:15.353 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:14 smithi193 ceph-mon[35962]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:15.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.731+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dfc02e560 con 0x7f1dfc1084d0 2024-05-14T17:00:15.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.732+0000 7f1df27fc700 1 -- 172.21.15.99:0/1556368065 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22243 (secure 0 0 0) 0x7f1df40aa350 con 0x7f1dfc1084d0 2024-05-14T17:00:15.382 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:15.391 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":80,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:00:11.851301+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:00:15.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1de80b2880 msgr2=0x7f1de80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1de80b2880 0x7f1de80b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007b60 tx=0x7f1df80073d0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 msgr2=0x7f1dfc104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc104b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007d70 tx=0x7f1df400a340 comp rx=0 tx=0).stop 2024-05-14T17:00:15.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 shutdown_connections 2024-05-14T17:00:15.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1de80b2880 0x7f1de80b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc1084d0 0x7f1dfc104b20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dfc1024d0 0x7f1dfc1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 --2- 172.21.15.99:0/1556368065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dfc102e90 0x7f1dfc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 >> 172.21.15.99:0/1556368065 conn(0x7f1dfc078490 msgr2=0x7f1dfc06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:15.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.734+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 shutdown_connections 2024-05-14T17:00:15.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:14.735+0000 7f1e038e5700 1 -- 172.21.15.99:0/1556368065 wait complete. 2024-05-14T17:00:15.624 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:15 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1556368065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:15.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:15 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1556368065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:15.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:15 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1556368065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:15.660 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:00:15.792 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:00:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: 2024-05-14T17:00:15.508+0000 7f62f1572700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T17:00:15.924 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:16.276 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:16 smithi099 ceph-mon[31851]: pgmap v286: 97 pgs: 19 creating+peering, 78 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-14T17:00:16.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:16 smithi150 ceph-mon[36204]: pgmap v286: 97 pgs: 19 creating+peering, 78 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-14T17:00:16.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:16 smithi193 ceph-mon[35962]: pgmap v286: 97 pgs: 19 creating+peering, 78 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-14T17:00:17.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 -- 172.21.15.99:0/3689251432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 msgr2=0x7f7f80110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 --2- 172.21.15.99:0/3689251432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f80110440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f780099c0 tx=0x7f7f78009cd0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 -- 172.21.15.99:0/3689251432 shutdown_connections 2024-05-14T17:00:17.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 --2- 172.21.15.99:0/3689251432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f80110440 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 --2- 172.21.15.99:0/3689251432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f80109670 0x7f7f80109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 --2- 172.21.15.99:0/3689251432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 0x7f7f80103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 -- 172.21.15.99:0/3689251432 >> 172.21.15.99:0/3689251432 conn(0x7f7f8007a730 msgr2=0x7f7f8007ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:17.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 -- 172.21.15.99:0/3689251432 shutdown_connections 2024-05-14T17:00:17.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.614+0000 7f7f86d09700 1 -- 172.21.15.99:0/3689251432 wait complete. 2024-05-14T17:00:17.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 Processor -- start 2024-05-14T17:00:17.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 -- start start 2024-05-14T17:00:17.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 0x7f7f8019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f8019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f80109670 0x7f7f801a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801137d0 con 0x7f7f80103f90 2024-05-14T17:00:17.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80113650 con 0x7f7f801035d0 2024-05-14T17:00:17.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f86d09700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80113950 con 0x7f7f80109670 2024-05-14T17:00:17.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f8019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f8019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39512/0 (socket says 172.21.15.99:39512) 2024-05-14T17:00:17.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 -- 172.21.15.99:0/2912789282 learned_addr learned my addr 172.21.15.99:0/2912789282 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:17.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f852a6700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f80109670 0x7f7f801a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f84aa5700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 0x7f7f8019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 -- 172.21.15.99:0/2912789282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f80109670 msgr2=0x7f7f801a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f80109670 0x7f7f801a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 -- 172.21.15.99:0/2912789282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 msgr2=0x7f7f8019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 0x7f7f8019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f7ffff700 1 -- 172.21.15.99:0/2912789282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f801a0f00 con 0x7f7f80103f90 2024-05-14T17:00:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.615+0000 7f7f84aa5700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 0x7f7f8019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:17.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.616+0000 7f7f7ffff700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f8019c430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800c920 tx=0x7f7f6800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:17.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.616+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68007b00 con 0x7f7f80103f90 2024-05-14T17:00:18.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.616+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f68004ce0 con 0x7f7f80103f90 2024-05-14T17:00:18.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.616+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f801a11f0 con 0x7f7f80103f90 2024-05-14T17:00:18.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.616+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68005650 con 0x7f7f80103f90 2024-05-14T17:00:18.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.616+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a8a70 con 0x7f7f80103f90 2024-05-14T17:00:18.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.617+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8006a230 con 0x7f7f80103f90 2024-05-14T17:00:18.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.618+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7f68004750 con 0x7f7f80103f90 2024-05-14T17:00:19.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.619+0000 7f7f7dffb700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7f6c0b2900 0x7f7f6c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:19.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.619+0000 7f7f84aa5700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7f6c0b2900 0x7f7f6c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:19.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.619+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7f680e1c20 con 0x7f7f80103f90 2024-05-14T17:00:19.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.619+0000 7f7f84aa5700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7f6c0b2900 0x7f7f6c0b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007eb0 tx=0x7f7f700073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.623+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f68014030 con 0x7f7f80103f90 2024-05-14T17:00:19.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.832+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f801a8d00 con 0x7f7f80103f90 2024-05-14T17:00:19.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.833+0000 7f7f7dffb700 1 -- 172.21.15.99:0/2912789282 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22243 (secure 0 0 0) 0x7f7f680a9a90 con 0x7f7f80103f90 2024-05-14T17:00:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:19.593 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":80,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:00:11.851301+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:00:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7f6c0b2900 msgr2=0x7f7f6c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:19.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7f6c0b2900 0x7f7f6c0b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007eb0 tx=0x7f7f700073c0 comp rx=0 tx=0).stop 2024-05-14T17:00:19.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 msgr2=0x7f7f8019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:19.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f8019c430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800c920 tx=0x7f7f6800cce0 comp rx=0 tx=0).stop 2024-05-14T17:00:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 shutdown_connections 2024-05-14T17:00:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7f6c0b2900 0x7f7f6c0b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:19.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f80109670 0x7f7f801a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:19.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f801035d0 0x7f7f8019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:19.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 --2- 172.21.15.99:0/2912789282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f80103f90 0x7f7f8019c430 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:19.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 >> 172.21.15.99:0/2912789282 conn(0x7f7f8007a730 msgr2=0x7f7f8010e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 shutdown_connections 2024-05-14T17:00:19.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:17.836+0000 7f7f86d09700 1 -- 172.21.15.99:0/2912789282 wait complete. 2024-05-14T17:00:19.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:18 smithi099 ceph-mon[31851]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-14T17:00:19.887 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:18 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2912789282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:19.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-14T17:00:19.905 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:18 smithi193 ceph-mon[35962]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-14T17:00:19.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:18 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2912789282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:19.930 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:18 smithi150 ceph-mon[36204]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-14T17:00:19.944 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:18 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2912789282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:20.211 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:20.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:20 smithi099 ceph-mon[31851]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T17:00:20.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:20 smithi150 ceph-mon[36204]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T17:00:20.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:20 smithi193 ceph-mon[35962]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-14T17:00:21.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 -- 172.21.15.99:0/2615163677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 msgr2=0x7fe9701020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:21.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2615163677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe9701020d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9600099c0 tx=0x7fe960009cd0 comp rx=0 tx=0).stop 2024-05-14T17:00:21.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 -- 172.21.15.99:0/2615163677 shutdown_connections 2024-05-14T17:00:21.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2615163677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe970108930 0x7fe9701104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:21.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2615163677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe9701020d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:21.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2615163677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9701026a0 0x7fe970108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:21.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 -- 172.21.15.99:0/2615163677 >> 172.21.15.99:0/2615163677 conn(0x7fe970078a40 msgr2=0x7fe9700ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:21.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 -- 172.21.15.99:0/2615163677 shutdown_connections 2024-05-14T17:00:21.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.789+0000 7fe974bcc700 1 -- 172.21.15.99:0/2615163677 wait complete. 2024-05-14T17:00:21.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 Processor -- start 2024-05-14T17:00:21.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 -- start start 2024-05-14T17:00:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe97019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:21.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9701026a0 0x7fe97019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:21.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe970108930 0x7fe9701999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:21.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe970113730 con 0x7fe970101cf0 2024-05-14T17:00:22.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701135b0 con 0x7fe970108930 2024-05-14T17:00:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe974bcc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9701138b0 con 0x7fe9701026a0 2024-05-14T17:00:22.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe97019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe97019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39540/0 (socket says 172.21.15.99:39540) 2024-05-14T17:00:22.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.790+0000 7fe96e59c700 1 -- 172.21.15.99:0/2568990853 learned_addr learned my addr 172.21.15.99:0/2568990853 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:22.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96ed9d700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe970108930 0x7fe9701999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe965d9b700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9701026a0 0x7fe97019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96e59c700 1 -- 172.21.15.99:0/2568990853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9701026a0 msgr2=0x7fe97019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:22.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96e59c700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9701026a0 0x7fe97019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96e59c700 1 -- 172.21.15.99:0/2568990853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe970108930 msgr2=0x7fe9701999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96e59c700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe970108930 0x7fe9701999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96e59c700 1 -- 172.21.15.99:0/2568990853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97019a130 con 0x7fe970101cf0 2024-05-14T17:00:22.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe96e59c700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe97019f810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe960009fd0 tx=0x7fe960005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960004830 con 0x7fe970101cf0 2024-05-14T17:00:22.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe960005020 con 0x7fe970101cf0 2024-05-14T17:00:22.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960003ef0 con 0x7fe970101cf0 2024-05-14T17:00:22.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97019a3c0 con 0x7fe970101cf0 2024-05-14T17:00:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.791+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701a66e0 con 0x7fe970101cf0 2024-05-14T17:00:22.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.792+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970104db0 con 0x7fe970101cf0 2024-05-14T17:00:22.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.794+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe960029060 con 0x7fe970101cf0 2024-05-14T17:00:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.795+0000 7fe967fff700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe95c0b2880 0x7fe95c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:22.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.795+0000 7fe965d9b700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe95c0b2880 0x7fe95c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:22.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.795+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe9600e2de0 con 0x7fe970101cf0 2024-05-14T17:00:22.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.795+0000 7fe965d9b700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe95c0b2880 0x7fe95c0b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe958005950 tx=0x7fe9580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:22.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:21.799+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe960013070 con 0x7fe970101cf0 2024-05-14T17:00:22.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.044+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe97002e560 con 0x7fe970101cf0 2024-05-14T17:00:22.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.085+0000 7fe967fff700 1 -- 172.21.15.99:0/2568990853 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fe9600aace0 con 0x7fe970101cf0 2024-05-14T17:00:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe95c0b2880 msgr2=0x7fe95c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe95c0b2880 0x7fe95c0b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe958005950 tx=0x7fe9580058e0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 msgr2=0x7fe97019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe97019f810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe960009fd0 tx=0x7fe960005f10 comp rx=0 tx=0).stop 2024-05-14T17:00:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 shutdown_connections 2024-05-14T17:00:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe95c0b2880 0x7fe95c0b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe970108930 0x7fe9701999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe970101cf0 0x7fe97019f810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 --2- 172.21.15.99:0/2568990853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9701026a0 0x7fe97019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:22.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 >> 172.21.15.99:0/2568990853 conn(0x7fe970078a40 msgr2=0x7fe9700ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:22.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 shutdown_connections 2024-05-14T17:00:22.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:22.088+0000 7fe974bcc700 1 -- 172.21.15.99:0/2568990853 wait complete. 2024-05-14T17:00:22.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:22 smithi193 ceph-mon[35962]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-14T17:00:22.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:22 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2568990853' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T17:00:22.586 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:22 smithi150 ceph-mon[36204]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-14T17:00:22.599 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:22 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2568990853' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T17:00:22.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:22 smithi099 ceph-mon[31851]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-14T17:00:22.620 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:22 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2568990853' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T17:00:22.718 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-14T17:00:22.733 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-14T17:00:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:23.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:23 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2568990853' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T17:00:23.358 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:23 smithi099 ceph-mon[31851]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:23 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2568990853' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T17:00:23.470 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:23 smithi150 ceph-mon[36204]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:23.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:23 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2568990853' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T17:00:23.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:23 smithi193 ceph-mon[35962]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:24.358 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:24 smithi099 ceph-mon[31851]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-14T17:00:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:24 smithi150 ceph-mon[36204]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-14T17:00:24.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:24 smithi193 ceph-mon[35962]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-14T17:00:24.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 -- 172.21.15.99:0/1102456826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 msgr2=0x7f721006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:24.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 --2- 172.21.15.99:0/1102456826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721006bf30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7204009990 tx=0x7f7204009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 -- 172.21.15.99:0/1102456826 shutdown_connections 2024-05-14T17:00:24.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 --2- 172.21.15.99:0/1102456826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006c470 0x7f72101103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:24.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 --2- 172.21.15.99:0/1102456826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72101095a0 0x7f7210109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:24.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 --2- 172.21.15.99:0/1102456826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721006bf30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:24.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 -- 172.21.15.99:0/1102456826 >> 172.21.15.99:0/1102456826 conn(0x7f72100793b0 msgr2=0x7f72100797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:24.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 -- 172.21.15.99:0/1102456826 shutdown_connections 2024-05-14T17:00:24.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.656+0000 7f7214917700 1 -- 172.21.15.99:0/1102456826 wait complete. 2024-05-14T17:00:24.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 Processor -- start 2024-05-14T17:00:24.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 -- start start 2024-05-14T17:00:24.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:24.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f721006c470 0x7f721019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:24.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72101095a0 0x7f7210199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:24.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72101136b0 con 0x7f721006bab0 2024-05-14T17:00:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7210113530 con 0x7f72101095a0 2024-05-14T17:00:25.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f7214917700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7210113830 con 0x7f721006c470 2024-05-14T17:00:25.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f720e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:25.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f720e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42280/0 (socket says 172.21.15.99:42280) 2024-05-14T17:00:25.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f720e59c700 1 -- 172.21.15.99:0/4185505155 learned_addr learned my addr 172.21.15.99:0/4185505155 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.657+0000 7f720ed9d700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72101095a0 0x7f7210199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:25.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720dd9b700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f721006c470 0x7f721019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:25.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720e59c700 1 -- 172.21.15.99:0/4185505155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f721006c470 msgr2=0x7f721019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720e59c700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f721006c470 0x7f721019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720e59c700 1 -- 172.21.15.99:0/4185505155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72101095a0 msgr2=0x7f7210199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:25.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720e59c700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72101095a0 0x7f7210199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720e59c700 1 -- 172.21.15.99:0/4185505155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721019a0b0 con 0x7f721006bab0 2024-05-14T17:00:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f720e59c700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721019f790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f720000c9b0 tx=0x7f720000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:25.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200007b00 con 0x7f721006bab0 2024-05-14T17:00:25.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7200007ca0 con 0x7f721006bab0 2024-05-14T17:00:25.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200018730 con 0x7f721006bab0 2024-05-14T17:00:25.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721019a3a0 con 0x7f721006bab0 2024-05-14T17:00:25.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.658+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72101a6660 con 0x7f721006bab0 2024-05-14T17:00:25.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.659+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721006a230 con 0x7f721006bab0 2024-05-14T17:00:25.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.660+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f720000f450 con 0x7f721006bab0 2024-05-14T17:00:25.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.661+0000 7f71ff7fe700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71f80b2900 0x7f71f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:25.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.661+0000 7f720dd9b700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71f80b2900 0x7f71f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:25.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.661+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f72000e2360 con 0x7f721006bab0 2024-05-14T17:00:25.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.662+0000 7f720dd9b700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71f80b2900 0x7f71f80b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7204005cd0 tx=0x7f7204005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:25.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.665+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7200014030 con 0x7f721006bab0 2024-05-14T17:00:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:24.912+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f721002e560 con 0x7f721006bab0 2024-05-14T17:00:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.103+0000 7f71ff7fe700 1 -- 172.21.15.99:0/4185505155 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f72000aa2e0 con 0x7f721006bab0 2024-05-14T17:00:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71f80b2900 msgr2=0x7f71f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71f80b2900 0x7f71f80b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7204005cd0 tx=0x7f7204005c40 comp rx=0 tx=0).stop 2024-05-14T17:00:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 msgr2=0x7f721019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721019f790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f720000c9b0 tx=0x7f720000cd70 comp rx=0 tx=0).stop 2024-05-14T17:00:25.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 shutdown_connections 2024-05-14T17:00:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71f80b2900 0x7f71f80b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72101095a0 0x7f7210199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f721006bab0 0x7f721019f790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 --2- 172.21.15.99:0/4185505155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f721006c470 0x7f721019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:25.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 >> 172.21.15.99:0/4185505155 conn(0x7f72100793b0 msgr2=0x7f7210101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:25.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 shutdown_connections 2024-05-14T17:00:25.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:25.106+0000 7f7214917700 1 -- 172.21.15.99:0/4185505155 wait complete. 2024-05-14T17:00:25.518 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:25 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4185505155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T17:00:25.525 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:25 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4185505155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T17:00:25.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:25 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4185505155' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T17:00:25.793 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-14T17:00:26.045 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:26.400 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T17:00:26.413 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T17:00:26.463 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4185505155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T17:00:26.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T17:00:26.552 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:26.564 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-14T17:00:26.578 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:26 smithi099 ceph-mon[31851]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-14T17:00:26.591 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T17:00:26.603 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T17:00:26.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4185505155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T17:00:26.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T17:00:26.629 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:26.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-14T17:00:26.648 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:26 smithi193 ceph-mon[35962]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-14T17:00:26.655 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T17:00:26.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T17:00:26.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4185505155' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T17:00:26.680 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T17:00:26.692 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-14T17:00:26.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-14T17:00:26.765 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:26 smithi150 ceph-mon[36204]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-14T17:00:27.391 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] up:active 2024-05-14T17:00:27.400 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T17:00:27.420 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T17:00:27.428 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: Cluster is now healthy 2024-05-14T17:00:27.437 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-14T17:00:27.446 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-14T17:00:27.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:27 smithi099 ceph-mon[31851]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-14T17:00:27.482 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] up:active 2024-05-14T17:00:27.491 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T17:00:27.501 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T17:00:27.515 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: Cluster is now healthy 2024-05-14T17:00:27.523 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-14T17:00:27.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-14T17:00:27.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:27 smithi150 ceph-mon[36204]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-14T17:00:27.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] up:active 2024-05-14T17:00:27.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T17:00:27.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T17:00:27.670 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: Cluster is now healthy 2024-05-14T17:00:27.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-14T17:00:27.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-14T17:00:27.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:27 smithi193 ceph-mon[35962]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-14T17:00:27.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 -- 172.21.15.99:0/1967581403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50102e00 msgr2=0x7fdd5010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 --2- 172.21.15.99:0/1967581403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50102e00 0x7fdd5010f2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b0a0 tx=0x7fdd4800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 -- 172.21.15.99:0/1967581403 shutdown_connections 2024-05-14T17:00:27.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 --2- 172.21.15.99:0/1967581403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50102e00 0x7fdd5010f2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 --2- 172.21.15.99:0/1967581403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50108520 0x7fdd50108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 --2- 172.21.15.99:0/1967581403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50102440 0x7fdd501028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 -- 172.21.15.99:0/1967581403 >> 172.21.15.99:0/1967581403 conn(0x7fdd500793f0 msgr2=0x7fdd50079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:27.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 -- 172.21.15.99:0/1967581403 shutdown_connections 2024-05-14T17:00:27.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.604+0000 7fdd54b25700 1 -- 172.21.15.99:0/1967581403 wait complete. 2024-05-14T17:00:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 Processor -- start 2024-05-14T17:00:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 -- start start 2024-05-14T17:00:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50102440 0x7fdd5019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50102e00 0x7fdd5019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 0x7fdd501a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501125f0 con 0x7fdd50108520 2024-05-14T17:00:28.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50112470 con 0x7fdd50102440 2024-05-14T17:00:28.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd54b25700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50112770 con 0x7fdd50102e00 2024-05-14T17:00:28.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 0x7fdd501a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 0x7fdd501a5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42294/0 (socket says 172.21.15.99:42294) 2024-05-14T17:00:28.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 -- 172.21.15.99:0/3585295384 learned_addr learned my addr 172.21.15.99:0/3585295384 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4dd9b700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50102e00 0x7fdd5019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4e59c700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50102440 0x7fdd5019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 -- 172.21.15.99:0/3585295384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50102e00 msgr2=0x7fdd5019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50102e00 0x7fdd5019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 -- 172.21.15.99:0/3585295384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50102440 msgr2=0x7fdd5019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50102440 0x7fdd5019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 -- 172.21.15.99:0/3585295384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd501a6010 con 0x7fdd50108520 2024-05-14T17:00:28.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4e59c700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50102440 0x7fdd5019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.605+0000 7fdd4ed9d700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 0x7fdd501a5ad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b830 tx=0x7fdd48004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:28.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.606+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48021070 con 0x7fdd50108520 2024-05-14T17:00:28.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.606+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd501a61e0 con 0x7fdd50108520 2024-05-14T17:00:28.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.606+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd48007e30 con 0x7fdd50108520 2024-05-14T17:00:28.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.606+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480047d0 con 0x7fdd50108520 2024-05-14T17:00:28.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.606+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd501a6640 con 0x7fdd50108520 2024-05-14T17:00:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.607+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5006a230 con 0x7fdd50108520 2024-05-14T17:00:28.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.608+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdd48007980 con 0x7fdd50108520 2024-05-14T17:00:28.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.609+0000 7fdd3f7fe700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdd380b2880 0x7fdd380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.609+0000 7fdd4e59c700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdd380b2880 0x7fdd380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.609+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fdd480e30d0 con 0x7fdd50108520 2024-05-14T17:00:28.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.610+0000 7fdd4e59c700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdd380b2880 0x7fdd380b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd40005e10 tx=0x7fdd40005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:28.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.613+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd480aafb0 con 0x7fdd50108520 2024-05-14T17:00:28.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:27.862+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fdd5002e560 con 0x7fdd50108520 2024-05-14T17:00:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.122+0000 7fdd3f7fe700 1 -- 172.21.15.99:0/3585295384 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fdd480aa8b0 con 0x7fdd50108520 2024-05-14T17:00:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdd380b2880 msgr2=0x7fdd380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdd380b2880 0x7fdd380b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd40005e10 tx=0x7fdd40005d60 comp rx=0 tx=0).stop 2024-05-14T17:00:28.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 msgr2=0x7fdd501a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 0x7fdd501a5ad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b830 tx=0x7fdd48004ce0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 shutdown_connections 2024-05-14T17:00:28.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdd380b2880 0x7fdd380b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd50108520 0x7fdd501a5ad0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd50102440 0x7fdd5019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 --2- 172.21.15.99:0/3585295384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50102e00 0x7fdd5019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 >> 172.21.15.99:0/3585295384 conn(0x7fdd500793f0 msgr2=0x7fdd50100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 shutdown_connections 2024-05-14T17:00:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:28.124+0000 7fdd54b25700 1 -- 172.21.15.99:0/3585295384 wait complete. 2024-05-14T17:00:28.587 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:28 smithi193 ceph-mon[35962]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-14T17:00:28.604 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:28 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] up:active 2024-05-14T17:00:28.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:28 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-14T17:00:28.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:28 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3585295384' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T17:00:28.641 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:28 smithi150 ceph-mon[36204]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-14T17:00:28.654 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:28 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] up:active 2024-05-14T17:00:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:28 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-14T17:00:28.687 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:28 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3585295384' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T17:00:28.707 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:28 smithi099 ceph-mon[31851]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 2 op/s 2024-05-14T17:00:28.732 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:28 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] up:active 2024-05-14T17:00:28.742 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:28 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-14T17:00:28.773 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:28 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3585295384' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T17:00:28.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-14T17:00:29.134 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:29 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3585295384' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T17:00:29.143 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:29 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-14T17:00:29.152 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:29 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:29.162 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:29 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3585295384' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T17:00:29.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:29 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-14T17:00:29.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:29 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:29.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:29 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3585295384' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T17:00:29.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:29 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-14T17:00:29.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:29 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:30.196 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:30 smithi099 ceph-mon[31851]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-14T17:00:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:30 smithi150 ceph-mon[36204]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-14T17:00:30.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:30 smithi193 ceph-mon[35962]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-14T17:00:30.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.704+0000 7fbe3f885700 1 -- 172.21.15.99:0/3718192133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 msgr2=0x7fbe381028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:30.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.704+0000 7fbe3f885700 1 --2- 172.21.15.99:0/3718192133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe381028c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009990 tx=0x7fbe34009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 -- 172.21.15.99:0/3718192133 shutdown_connections 2024-05-14T17:00:30.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 --2- 172.21.15.99:0/3718192133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe38102e00 0x7fbe3810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 --2- 172.21.15.99:0/3718192133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe38108520 0x7fbe38108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 --2- 172.21.15.99:0/3718192133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe381028c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 -- 172.21.15.99:0/3718192133 >> 172.21.15.99:0/3718192133 conn(0x7fbe380793f0 msgr2=0x7fbe38079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 -- 172.21.15.99:0/3718192133 shutdown_connections 2024-05-14T17:00:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 -- 172.21.15.99:0/3718192133 wait complete. 2024-05-14T17:00:30.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 Processor -- start 2024-05-14T17:00:30.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.705+0000 7fbe3f885700 1 -- start start 2024-05-14T17:00:30.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe3819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe38102e00 0x7fbe3819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:30.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe38108520 0x7fbe38199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:30.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381125d0 con 0x7fbe38102440 2024-05-14T17:00:30.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112450 con 0x7fbe38108520 2024-05-14T17:00:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112750 con 0x7fbe38102e00 2024-05-14T17:00:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe3819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe3819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42318/0 (socket says 172.21.15.99:42318) 2024-05-14T17:00:30.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 -- 172.21.15.99:0/2945675137 learned_addr learned my addr 172.21.15.99:0/2945675137 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:30.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3ce20700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe38102e00 0x7fbe3819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:30.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3de22700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe38108520 0x7fbe38199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:30.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 -- 172.21.15.99:0/2945675137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe38102e00 msgr2=0x7fbe3819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:30.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe38102e00 0x7fbe3819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 -- 172.21.15.99:0/2945675137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe38108520 msgr2=0x7fbe38199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:30.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe38108520 0x7fbe38199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 -- 172.21.15.99:0/2945675137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3819a090 con 0x7fbe38102440 2024-05-14T17:00:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3d621700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe3819f770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c9b0 tx=0x7fbe2800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:31.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007b00 con 0x7fbe38102440 2024-05-14T17:00:31.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe28007ca0 con 0x7fbe38102440 2024-05-14T17:00:31.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.706+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3819a380 con 0x7fbe38102440 2024-05-14T17:00:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.707+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28018660 con 0x7fbe38102440 2024-05-14T17:00:31.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.707+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381a6640 con 0x7fbe38102440 2024-05-14T17:00:31.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.707+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3806a230 con 0x7fbe38102440 2024-05-14T17:00:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.709+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbe2800f450 con 0x7fbe38102440 2024-05-14T17:00:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.709+0000 7fbe2e7fc700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe240b2880 0x7fbe240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.709+0000 7fbe3ce20700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe240b2880 0x7fbe240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:31.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.710+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fbe280e22d0 con 0x7fbe38102440 2024-05-14T17:00:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.710+0000 7fbe3ce20700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe240b2880 0x7fbe240b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b5c0 tx=0x7fbe34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.713+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe28014030 con 0x7fbe38102440 2024-05-14T17:00:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:30.961+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fbe3802e560 con 0x7fbe38102440 2024-05-14T17:00:31.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.143+0000 7fbe2e7fc700 1 -- 172.21.15.99:0/2945675137 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fbe280aa1d0 con 0x7fbe38102440 2024-05-14T17:00:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe240b2880 msgr2=0x7fbe240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:31.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe240b2880 0x7fbe240b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b5c0 tx=0x7fbe34005960 comp rx=0 tx=0).stop 2024-05-14T17:00:31.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 msgr2=0x7fbe3819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:31.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe3819f770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c9b0 tx=0x7fbe2800ccc0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 shutdown_connections 2024-05-14T17:00:31.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe240b2880 0x7fbe240b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe38108520 0x7fbe38199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe38102440 0x7fbe3819f770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 --2- 172.21.15.99:0/2945675137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe38102e00 0x7fbe3819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:31.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 >> 172.21.15.99:0/2945675137 conn(0x7fbe380793f0 msgr2=0x7fbe3806b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 shutdown_connections 2024-05-14T17:00:31.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:31.146+0000 7fbe3f885700 1 -- 172.21.15.99:0/2945675137 wait complete. 2024-05-14T17:00:31.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:31 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2945675137' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T17:00:31.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:31 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2945675137' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T17:00:31.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:31 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2945675137' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T17:00:31.805 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-14T17:00:31.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-14T17:00:32.065 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:32.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:32 smithi150 ceph-mon[36204]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-14T17:00:32.465 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:32 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2945675137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T17:00:32.471 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:32 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:32.482 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:32 smithi099 ceph-mon[31851]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-14T17:00:32.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:32 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2945675137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T17:00:32.517 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:32 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:32.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:32 smithi193 ceph-mon[35962]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-14T17:00:32.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:32 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2945675137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T17:00:32.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:32 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:33.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 -- 172.21.15.99:0/3874671705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006bbe0 msgr2=0x7f14901103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:33.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 --2- 172.21.15.99:0/3874671705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006bbe0 0x7f14901103d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f148800b0a0 tx=0x7f148800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 -- 172.21.15.99:0/3874671705 shutdown_connections 2024-05-14T17:00:33.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 --2- 172.21.15.99:0/3874671705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006bbe0 0x7f14901103d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 --2- 172.21.15.99:0/3874671705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14901095b0 0x7f1490109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 --2- 172.21.15.99:0/3874671705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f149006b220 0x7f149006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 -- 172.21.15.99:0/3874671705 >> 172.21.15.99:0/3874671705 conn(0x7f14900793f0 msgr2=0x7f1490079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:33.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 -- 172.21.15.99:0/3874671705 shutdown_connections 2024-05-14T17:00:33.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 -- 172.21.15.99:0/3874671705 wait complete. 2024-05-14T17:00:33.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 Processor -- start 2024-05-14T17:00:33.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.658+0000 7f149795a700 1 -- start start 2024-05-14T17:00:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 0x7f149019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:33.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f149006bbe0 0x7f149019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:33.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14901095b0 0x7f14901999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:33.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14901136f0 con 0x7f149006b220 2024-05-14T17:00:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1490113570 con 0x7f149006bbe0 2024-05-14T17:00:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1490113870 con 0x7f14901095b0 2024-05-14T17:00:33.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 0x7f149019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:33.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 0x7f149019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42336/0 (socket says 172.21.15.99:42336) 2024-05-14T17:00:33.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 -- 172.21.15.99:0/3747026525 learned_addr learned my addr 172.21.15.99:0/3747026525 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:33.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 -- 172.21.15.99:0/3747026525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14901095b0 msgr2=0x7f14901999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:33.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14901095b0 0x7f14901999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 -- 172.21.15.99:0/3747026525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f149006bbe0 msgr2=0x7f149019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:33.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f149006bbe0 0x7f149019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:33.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 -- 172.21.15.99:0/3747026525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149019a150 con 0x7f149006b220 2024-05-14T17:00:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14956f6700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 0x7f149019f7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e9a0 tx=0x7f148c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:33.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c00cbd0 con 0x7f149006b220 2024-05-14T17:00:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f148c00cd70 con 0x7f149006b220 2024-05-14T17:00:34.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149019a440 con 0x7f149006b220 2024-05-14T17:00:34.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c018680 con 0x7f149006b220 2024-05-14T17:00:34.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.659+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14901a66b0 con 0x7f149006b220 2024-05-14T17:00:34.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.660+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149006a230 con 0x7f149006b220 2024-05-14T17:00:34.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.662+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f148c010970 con 0x7f149006b220 2024-05-14T17:00:34.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.663+0000 7f14867fc700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f147c0b2900 0x7f147c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:34.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.663+0000 7f1494ef5700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f147c0b2900 0x7f147c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:34.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.663+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f148c0e2800 con 0x7f149006b220 2024-05-14T17:00:34.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.664+0000 7f1494ef5700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f147c0b2900 0x7f147c0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1480005950 tx=0x7f14800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.666+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f148c0aa670 con 0x7f149006b220 2024-05-14T17:00:34.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.900+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f149002e5d0 con 0x7f147c0b2900 2024-05-14T17:00:34.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.924+0000 7f14867fc700 1 -- 172.21.15.99:0/3747026525 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f149002e5d0 con 0x7f147c0b2900 2024-05-14T17:00:34.161 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:00:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:00:33.905+0000 7f3f9cb02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:00:34.175 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:00:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:00:33.905+0000 7f3f9cb02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:00:34.190 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:00:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:00:33.905+0000 7f3f9cb02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:00:34.200 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:00:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:00:33.905+0000 7f3f9cb02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:00:34.211 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:00:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:00:33.905+0000 7f3f9cb02700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:00:34.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f147c0b2900 msgr2=0x7f147c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:34.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f147c0b2900 0x7f147c0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1480005950 tx=0x7f14800058e0 comp rx=0 tx=0).stop 2024-05-14T17:00:34.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 msgr2=0x7f149019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:34.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 0x7f149019f7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e9a0 tx=0x7f148c00ed60 comp rx=0 tx=0).stop 2024-05-14T17:00:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 shutdown_connections 2024-05-14T17:00:34.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f147c0b2900 0x7f147c0b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:34.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14901095b0 0x7f14901999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:34.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f149006b220 0x7f149019f7e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:34.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 --2- 172.21.15.99:0/3747026525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f149006bbe0 0x7f149019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:34.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.926+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 >> 172.21.15.99:0/3747026525 conn(0x7f14900793f0 msgr2=0x7f1490101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:34.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.927+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 shutdown_connections 2024-05-14T17:00:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:33.927+0000 7f149795a700 1 -- 172.21.15.99:0/3747026525 wait complete. 2024-05-14T17:00:34.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:34 smithi099 ceph-mon[31851]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-14T17:00:34.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:34 smithi150 ceph-mon[36204]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-14T17:00:34.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:34 smithi193 ceph-mon[35962]: pgmap v295: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-14T17:00:34.690 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-14T17:00:34.941 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:35.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:35 smithi099 ceph-mon[31851]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:35.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:35 smithi150 ceph-mon[36204]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:35 smithi193 ceph-mon[35962]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:36.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:36 smithi099 ceph-mon[31851]: pgmap v296: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:36.542 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:36 smithi099 ceph-mon[31851]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T17:00:36.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:36 smithi193 ceph-mon[35962]: pgmap v296: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:36.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:36 smithi193 ceph-mon[35962]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T17:00:36.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:36 smithi150 ceph-mon[36204]: pgmap v296: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:36.577 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:36 smithi150 ceph-mon[36204]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T17:00:36.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 -- 172.21.15.99:0/3083666038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 msgr2=0x7fcefc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:36.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 --2- 172.21.15.99:0/3083666038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc108900 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fceec009990 tx=0x7fceec009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 -- 172.21.15.99:0/3083666038 shutdown_connections 2024-05-14T17:00:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 --2- 172.21.15.99:0/3083666038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 0x7fcefc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 --2- 172.21.15.99:0/3083666038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc108900 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 --2- 172.21.15.99:0/3083666038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc102440 0x7fcefc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 -- 172.21.15.99:0/3083666038 >> 172.21.15.99:0/3083666038 conn(0x7fcefc0793f0 msgr2=0x7fcefc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 -- 172.21.15.99:0/3083666038 shutdown_connections 2024-05-14T17:00:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 -- 172.21.15.99:0/3083666038 wait complete. 2024-05-14T17:00:36.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 Processor -- start 2024-05-14T17:00:36.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.574+0000 7fcf02736700 1 -- start start 2024-05-14T17:00:36.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf02736700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc102440 0x7fcefc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:36.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf02736700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 0x7fcefc19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:36.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf02736700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:36.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf02736700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc112650 con 0x7fcefc108520 2024-05-14T17:00:36.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf02736700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1124d0 con 0x7fcefc102e00 2024-05-14T17:00:36.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf02736700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1127d0 con 0x7fcefc102440 2024-05-14T17:00:36.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:36.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc1a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58144/0 (socket says 172.21.15.99:58144) 2024-05-14T17:00:36.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 -- 172.21.15.99:0/4022694366 learned_addr learned my addr 172.21.15.99:0/4022694366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:36.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcefbfff700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc102440 0x7fcefc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcef3fff700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 0x7fcefc19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:36.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 -- 172.21.15.99:0/4022694366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc102440 msgr2=0x7fcefc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:36.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc102440 0x7fcefc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 -- 172.21.15.99:0/4022694366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 msgr2=0x7fcefc19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 0x7fcefc19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 -- 172.21.15.99:0/4022694366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc1a5f70 con 0x7fcefc108520 2024-05-14T17:00:36.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcef3fff700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 0x7fcefc19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcf00cd3700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc1a5a30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcef400b970 tx=0x7fcef400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:36.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.575+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4019070 con 0x7fcefc108520 2024-05-14T17:00:36.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.576+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4014440 con 0x7fcefc108520 2024-05-14T17:00:36.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.576+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc1a6200 con 0x7fcefc108520 2024-05-14T17:00:37.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.576+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef400ce60 con 0x7fcefc108520 2024-05-14T17:00:37.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.576+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1a66c0 con 0x7fcefc108520 2024-05-14T17:00:37.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.576+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcefc10a9b0 con 0x7fcefc108520 2024-05-14T17:00:37.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.578+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcef401c070 con 0x7fcefc108520 2024-05-14T17:00:37.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.579+0000 7fcef9ffb700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcee80b2910 0x7fcee80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.579+0000 7fcefbfff700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcee80b2910 0x7fcee80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:37.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.579+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcef40e1a30 con 0x7fcefc108520 2024-05-14T17:00:37.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.579+0000 7fcefbfff700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcee80b2910 0x7fcee80b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fceec006010 tx=0x7fceec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.583+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcef40a9830 con 0x7fcefc108520 2024-05-14T17:00:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.820+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcefc063980 con 0x7fcee80b2910 2024-05-14T17:00:37.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.848+0000 7fcef9ffb700 1 -- 172.21.15.99:0/4022694366 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcefc063980 con 0x7fcee80b2910 2024-05-14T17:00:37.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcee80b2910 msgr2=0x7fcee80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcee80b2910 0x7fcee80b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fceec006010 tx=0x7fceec01a040 comp rx=0 tx=0).stop 2024-05-14T17:00:37.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 msgr2=0x7fcefc1a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:37.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc1a5a30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcef400b970 tx=0x7fcef400bd30 comp rx=0 tx=0).stop 2024-05-14T17:00:37.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 shutdown_connections 2024-05-14T17:00:37.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcee80b2910 0x7fcee80b4dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcefc108520 0x7fcefc1a5a30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc102440 0x7fcefc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 --2- 172.21.15.99:0/4022694366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcefc102e00 0x7fcefc19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.850+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 >> 172.21.15.99:0/4022694366 conn(0x7fcefc0793f0 msgr2=0x7fcefc101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:37.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.851+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 shutdown_connections 2024-05-14T17:00:37.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:36.851+0000 7fcf02736700 1 -- 172.21.15.99:0/4022694366 wait complete. 2024-05-14T17:00:37.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-14T17:00:37.820 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:38.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:38 smithi099 ceph-mon[31851]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:38.270 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:38 smithi099 ceph-mon[31851]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:38.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:38 smithi150 ceph-mon[36204]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:38.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:38 smithi150 ceph-mon[36204]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:38.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:38 smithi193 ceph-mon[35962]: pgmap v297: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:38.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:38 smithi193 ceph-mon[35962]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 -- 172.21.15.99:0/1924549259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 msgr2=0x7ff87c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 --2- 172.21.15.99:0/1924549259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c10f2e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff874009990 tx=0x7ff874009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 -- 172.21.15.99:0/1924549259 shutdown_connections 2024-05-14T17:00:39.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 --2- 172.21.15.99:0/1924549259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c10f2e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 --2- 172.21.15.99:0/1924549259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff87c108510 0x7ff87c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 --2- 172.21.15.99:0/1924549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 0x7ff87c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 -- 172.21.15.99:0/1924549259 >> 172.21.15.99:0/1924549259 conn(0x7ff87c0793b0 msgr2=0x7ff87c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 -- 172.21.15.99:0/1924549259 shutdown_connections 2024-05-14T17:00:39.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.447+0000 7ff883f96700 1 -- 172.21.15.99:0/1924549259 wait complete. 2024-05-14T17:00:39.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 Processor -- start 2024-05-14T17:00:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 -- start start 2024-05-14T17:00:39.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 0x7ff87c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff87c108510 0x7ff87c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1126c0 con 0x7ff87c102df0 2024-05-14T17:00:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c112540 con 0x7ff87c108510 2024-05-14T17:00:39.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff883f96700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c112840 con 0x7ff87c102430 2024-05-14T17:00:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58164/0 (socket says 172.21.15.99:58164) 2024-05-14T17:00:39.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 -- 172.21.15.99:0/2895737700 learned_addr learned my addr 172.21.15.99:0/2895737700 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881d32700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 0x7ff87c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:39.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 -- 172.21.15.99:0/2895737700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 msgr2=0x7ff87c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:39.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 0x7ff87c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 -- 172.21.15.99:0/2895737700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff87c108510 msgr2=0x7ff87c1989e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T17:00:39.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff87c108510 0x7ff87c1989e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 -- 172.21.15.99:0/2895737700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c199140 con 0x7ff87c102df0 2024-05-14T17:00:39.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881d32700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 0x7ff87c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:39.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.448+0000 7ff881531700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c19ed60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff87000b530 tx=0x7ff87000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:39.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.449+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870015020 con 0x7ff87c102df0 2024-05-14T17:00:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.449+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c199430 con 0x7ff87c102df0 2024-05-14T17:00:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.449+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff870011ce0 con 0x7ff87c102df0 2024-05-14T17:00:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.449+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870010d20 con 0x7ff87c102df0 2024-05-14T17:00:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.449+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1a56f0 con 0x7ff87c102df0 2024-05-14T17:00:39.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.450+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c10a9a0 con 0x7ff87c102df0 2024-05-14T17:00:39.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.451+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff870018020 con 0x7ff87c102df0 2024-05-14T17:00:39.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.452+0000 7ff87affd700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8640b2910 0x7ff8640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:39.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.452+0000 7ff881d32700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8640b2910 0x7ff8640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.452+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff8700e1920 con 0x7ff87c102df0 2024-05-14T17:00:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.452+0000 7ff881d32700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8640b2910 0x7ff8640b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff87c199b20 tx=0x7ff86c009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.456+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff87001f020 con 0x7ff87c102df0 2024-05-14T17:00:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.694+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff87c063980 con 0x7ff8640b2910 2024-05-14T17:00:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.700+0000 7ff87affd700 1 -- 172.21.15.99:0/2895737700 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff87c063980 con 0x7ff8640b2910 2024-05-14T17:00:39.910 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/qa/sv_0/891e29ac-691c-4c25-9a14-dd0fc36f14ed 2024-05-14T17:00:39.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8640b2910 msgr2=0x7ff8640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8640b2910 0x7ff8640b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff87c199b20 tx=0x7ff86c009570 comp rx=0 tx=0).stop 2024-05-14T17:00:39.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 msgr2=0x7ff87c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c19ed60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff87000b530 tx=0x7ff87000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 shutdown_connections 2024-05-14T17:00:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff8640b2910 0x7ff8640b4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff87c108510 0x7ff87c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:39.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff87c102430 0x7ff87c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 --2- 172.21.15.99:0/2895737700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff87c102df0 0x7ff87c19ed60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 >> 172.21.15.99:0/2895737700 conn(0x7ff87c0793b0 msgr2=0x7ff87c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:40.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 shutdown_connections 2024-05-14T17:00:40.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:39.702+0000 7ff883f96700 1 -- 172.21.15.99:0/2895737700 wait complete. 2024-05-14T17:00:40.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:40 smithi150 ceph-mon[36204]: pgmap v298: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:40.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:40 smithi099 ceph-mon[31851]: pgmap v298: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:40.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:40 smithi193 ceph-mon[35962]: pgmap v298: 97 pgs: 97 active+clean; 588 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-14T17:00:40.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-14T17:00:40.823 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:41.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:41 smithi150 ceph-mon[36204]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:41 smithi099 ceph-mon[31851]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:41 smithi193 ceph-mon[35962]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 -- 172.21.15.99:0/4245129036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb806bc90 msgr2=0x7fcdb806c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/4245129036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb806bc90 0x7fcdb806c110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 -- 172.21.15.99:0/4245129036 shutdown_connections 2024-05-14T17:00:42.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/4245129036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb806c650 0x7fcdb81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/4245129036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb81095d0 0x7fcdb81099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/4245129036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb806bc90 0x7fcdb806c110 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 -- 172.21.15.99:0/4245129036 >> 172.21.15.99:0/4245129036 conn(0x7fcdb80793e0 msgr2=0x7fcdb80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:42.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 -- 172.21.15.99:0/4245129036 shutdown_connections 2024-05-14T17:00:42.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.398+0000 7fcdbcb88700 1 -- 172.21.15.99:0/4245129036 wait complete. 2024-05-14T17:00:42.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 Processor -- start 2024-05-14T17:00:42.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 -- start start 2024-05-14T17:00:42.647 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:42 smithi150 ceph-mon[36204]: pgmap v299: 97 pgs: 97 active+clean; 602 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-14T17:00:42.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:42 smithi193 ceph-mon[35962]: pgmap v299: 97 pgs: 97 active+clean; 602 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-14T17:00:42.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb806bc90 0x7fcdb819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb806c650 0x7fcdb819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 0x7fcdb81999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8113770 con 0x7fcdb81095d0 2024-05-14T17:00:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81135f0 con 0x7fcdb806c650 2024-05-14T17:00:42.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdbcb88700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81138f0 con 0x7fcdb806bc90 2024-05-14T17:00:42.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 0x7fcdb81999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 0x7fcdb81999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58184/0 (socket says 172.21.15.99:58184) 2024-05-14T17:00:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 -- 172.21.15.99:0/3100170234 learned_addr learned my addr 172.21.15.99:0/3100170234 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:42.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb5d9b700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb806c650 0x7fcdb819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb659c700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb806bc90 0x7fcdb819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 -- 172.21.15.99:0/3100170234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb806bc90 msgr2=0x7fcdb819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb806bc90 0x7fcdb819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 -- 172.21.15.99:0/3100170234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb806c650 msgr2=0x7fcdb819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb806c650 0x7fcdb819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.399+0000 7fcdb6d9d700 1 -- 172.21.15.99:0/3100170234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb819a150 con 0x7fcdb81095d0 2024-05-14T17:00:42.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.400+0000 7fcdb6d9d700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 0x7fcdb81999f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b970 tx=0x7fcdb000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:42.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.400+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0019070 con 0x7fcdb81095d0 2024-05-14T17:00:42.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.400+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb000cde0 con 0x7fcdb81095d0 2024-05-14T17:00:42.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.400+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb819a440 con 0x7fcdb81095d0 2024-05-14T17:00:42.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.400+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0007370 con 0x7fcdb81095d0 2024-05-14T17:00:42.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.400+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb81a6700 con 0x7fcdb81095d0 2024-05-14T17:00:42.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.401+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb8104c70 con 0x7fcdb81095d0 2024-05-14T17:00:42.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.402+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcdb001c070 con 0x7fcdb81095d0 2024-05-14T17:00:42.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.403+0000 7fcda77fe700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcda00b2910 0x7fcda00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.403+0000 7fcdb659c700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcda00b2910 0x7fcda00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.403+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcdb00e1d40 con 0x7fcdb81095d0 2024-05-14T17:00:42.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.403+0000 7fcdb659c700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcda00b2910 0x7fcda00b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcda8005ea0 tx=0x7fcda8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:43.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.406+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcdb0023070 con 0x7fcdb81095d0 2024-05-14T17:00:43.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:42 smithi099 ceph-mon[31851]: pgmap v299: 97 pgs: 97 active+clean; 602 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-14T17:00:43.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.643+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcdb8063980 con 0x7fcda00b2910 2024-05-14T17:00:43.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.668+0000 7fcda77fe700 1 -- 172.21.15.99:0/3100170234 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcdb8063980 con 0x7fcda00b2910 2024-05-14T17:00:43.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcda00b2910 msgr2=0x7fcda00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:43.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcda00b2910 0x7fcda00b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcda8005ea0 tx=0x7fcda8005e30 comp rx=0 tx=0).stop 2024-05-14T17:00:43.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 msgr2=0x7fcdb81999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:43.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 0x7fcdb81999f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b970 tx=0x7fcdb000bc80 comp rx=0 tx=0).stop 2024-05-14T17:00:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 shutdown_connections 2024-05-14T17:00:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcda00b2910 0x7fcda00b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb81095d0 0x7fcdb81999f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb806bc90 0x7fcdb819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 --2- 172.21.15.99:0/3100170234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdb806c650 0x7fcdb819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 >> 172.21.15.99:0/3100170234 conn(0x7fcdb80793e0 msgr2=0x7fcdb8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:43.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 shutdown_connections 2024-05-14T17:00:43.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:42.671+0000 7fcdbcb88700 1 -- 172.21.15.99:0/3100170234 wait complete. 2024-05-14T17:00:43.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-14T17:00:43.622 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:44.410 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:44 smithi099 ceph-mon[31851]: pgmap v300: 97 pgs: 97 active+clean; 602 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-05-14T17:00:44.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:44 smithi099 ceph-mon[31851]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:44.675 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:44 smithi193 ceph-mon[35962]: pgmap v300: 97 pgs: 97 active+clean; 602 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-05-14T17:00:44.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:44 smithi193 ceph-mon[35962]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:44.696 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:44 smithi150 ceph-mon[36204]: pgmap v300: 97 pgs: 97 active+clean; 602 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-05-14T17:00:44.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:44 smithi150 ceph-mon[36204]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:45.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 -- 172.21.15.99:0/2797470471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 msgr2=0x7fc1a8108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:45.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 --2- 172.21.15.99:0/2797470471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a8108890 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc198009990 tx=0x7fc198009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 -- 172.21.15.99:0/2797470471 shutdown_connections 2024-05-14T17:00:45.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 --2- 172.21.15.99:0/2797470471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1a8102e30 0x7fc1a810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 --2- 172.21.15.99:0/2797470471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a8108890 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 --2- 172.21.15.99:0/2797470471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 0x7fc1a81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 -- 172.21.15.99:0/2797470471 >> 172.21.15.99:0/2797470471 conn(0x7fc1a80793f0 msgr2=0x7fc1a8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:45.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 -- 172.21.15.99:0/2797470471 shutdown_connections 2024-05-14T17:00:45.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 -- 172.21.15.99:0/2797470471 wait complete. 2024-05-14T17:00:45.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.365+0000 7fc1ad654700 1 Processor -- start 2024-05-14T17:00:45.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 -- start start 2024-05-14T17:00:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 0x7fc1a8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1a8102e30 0x7fc1a8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81125e0 con 0x7fc1a81084b0 2024-05-14T17:00:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8112460 con 0x7fc1a8102470 2024-05-14T17:00:45.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1ad654700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8112760 con 0x7fc1a8102e30 2024-05-14T17:00:45.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58300/0 (socket says 172.21.15.99:58300) 2024-05-14T17:00:45.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 -- 172.21.15.99:0/3181198964 learned_addr learned my addr 172.21.15.99:0/3181198964 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:45.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a67fc700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1a8102e30 0x7fc1a8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a6ffd700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 0x7fc1a8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 -- 172.21.15.99:0/3181198964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1a8102e30 msgr2=0x7fc1a8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:45.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1a8102e30 0x7fc1a8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 -- 172.21.15.99:0/3181198964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 msgr2=0x7fc1a8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:45.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 0x7fc1a8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a77fe700 1 -- 172.21.15.99:0/3181198964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81a80e0 con 0x7fc1a81084b0 2024-05-14T17:00:45.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.366+0000 7fc1a6ffd700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 0x7fc1a8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:45.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.367+0000 7fc1a77fe700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a81992f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0007ee0 tx=0x7fc1a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:45.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.367+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000dbd0 con 0x7fc1a81084b0 2024-05-14T17:00:45.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.367+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a000dd70 con 0x7fc1a81084b0 2024-05-14T17:00:45.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.367+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81a83d0 con 0x7fc1a81084b0 2024-05-14T17:00:45.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.367+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0005990 con 0x7fc1a81084b0 2024-05-14T17:00:45.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.367+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81a8910 con 0x7fc1a81084b0 2024-05-14T17:00:45.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.368+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a806a230 con 0x7fc1a81084b0 2024-05-14T17:00:45.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.369+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc1a001a020 con 0x7fc1a81084b0 2024-05-14T17:00:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.370+0000 7fc18ffff700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc1900b2910 0x7fc1900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.370+0000 7fc1a6ffd700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc1900b2910 0x7fc1900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.370+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc1a00e18d0 con 0x7fc1a81084b0 2024-05-14T17:00:45.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.370+0000 7fc1a6ffd700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc1900b2910 0x7fc1900b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc198006010 tx=0x7fc19800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:45.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.374+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1a0010070 con 0x7fc1a81084b0 2024-05-14T17:00:45.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.612+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc1a8063980 con 0x7fc1900b2910 2024-05-14T17:00:45.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.617+0000 7fc18ffff700 1 -- 172.21.15.99:0/3181198964 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc1a8063980 con 0x7fc1900b2910 2024-05-14T17:00:45.970 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/qa/sv_1/74f2e792-4109-459b-a6e4-7da494b4c0cd 2024-05-14T17:00:45.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc1900b2910 msgr2=0x7fc1900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc1900b2910 0x7fc1900b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc198006010 tx=0x7fc19800b540 comp rx=0 tx=0).stop 2024-05-14T17:00:46.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 msgr2=0x7fc1a81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:46.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a81992f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0007ee0 tx=0x7fc1a000d3b0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 shutdown_connections 2024-05-14T17:00:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fc1900b2910 0x7fc1900b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1a81084b0 0x7fc1a81992f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1a8102470 0x7fc1a8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 --2- 172.21.15.99:0/3181198964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1a8102e30 0x7fc1a8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 >> 172.21.15.99:0/3181198964 conn(0x7fc1a80793f0 msgr2=0x7fc1a8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:46.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 shutdown_connections 2024-05-14T17:00:46.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:45.620+0000 7fc1ad654700 1 -- 172.21.15.99:0/3181198964 wait complete. 2024-05-14T17:00:46.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:46 smithi099 ceph-mon[31851]: pgmap v301: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-14T17:00:46.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:00:46.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:46 smithi150 ceph-mon[36204]: pgmap v301: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-14T17:00:46.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:46 smithi193 ceph-mon[35962]: pgmap v301: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-14T17:00:46.643 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:47.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:47 smithi150 ceph-mon[36204]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:47.472 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:47 smithi099 ceph-mon[31851]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:47 smithi193 ceph-mon[35962]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:00:48.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 -- 172.21.15.99:0/623473750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 msgr2=0x7f10d006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:48.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 --2- 172.21.15.99:0/623473750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d006b6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009990 tx=0x7f10b8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 -- 172.21.15.99:0/623473750 shutdown_connections 2024-05-14T17:00:48.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 --2- 172.21.15.99:0/623473750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10d006bbe0 0x7f10d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 --2- 172.21.15.99:0/623473750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d01095b0 0x7f10d0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 --2- 172.21.15.99:0/623473750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d006b6a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 -- 172.21.15.99:0/623473750 >> 172.21.15.99:0/623473750 conn(0x7f10d00793f0 msgr2=0x7f10d0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:48.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 -- 172.21.15.99:0/623473750 shutdown_connections 2024-05-14T17:00:48.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.368+0000 7f10d4f00700 1 -- 172.21.15.99:0/623473750 wait complete. 2024-05-14T17:00:48.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 Processor -- start 2024-05-14T17:00:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 -- start start 2024-05-14T17:00:48.502 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:48 smithi150 ceph-mon[36204]: pgmap v302: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-14T17:00:48.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:48.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d006bbe0 0x7f10d019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:48.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10d01095b0 0x7f10d0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:48.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0113740 con 0x7f10d006b220 2024-05-14T17:00:48.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01135c0 con 0x7f10d01095b0 2024-05-14T17:00:48.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.369+0000 7f10d4f00700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01138c0 con 0x7f10d006bbe0 2024-05-14T17:00:48.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:48.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58330/0 (socket says 172.21.15.99:58330) 2024-05-14T17:00:48.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 -- 172.21.15.99:0/2609758856 learned_addr learned my addr 172.21.15.99:0/2609758856 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ced9d700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10d01095b0 0x7f10d0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10c7fff700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d006bbe0 0x7f10d019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:48.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 -- 172.21.15.99:0/2609758856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d006bbe0 msgr2=0x7f10d019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d006bbe0 0x7f10d019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 -- 172.21.15.99:0/2609758856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10d01095b0 msgr2=0x7f10d0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:48.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10d01095b0 0x7f10d0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:48.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 -- 172.21.15.99:0/2609758856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d019a260 con 0x7f10d006b220 2024-05-14T17:00:48.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10c7fff700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d006bbe0 0x7f10d019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10ce59c700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d019f940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10c000d980 tx=0x7f10c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:48.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0017070 con 0x7f10d006b220 2024-05-14T17:00:48.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c0005020 con 0x7f10d006b220 2024-05-14T17:00:48.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0009b50 con 0x7f10d006b220 2024-05-14T17:00:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d019a550 con 0x7f10d006b220 2024-05-14T17:00:48.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.370+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01a6810 con 0x7f10d006b220 2024-05-14T17:00:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.371+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d006a230 con 0x7f10d006b220 2024-05-14T17:00:48.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.373+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f10c00051d0 con 0x7f10d006b220 2024-05-14T17:00:48.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.374+0000 7f10c77fe700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f10bc0b2990 0x7f10bc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:48.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.374+0000 7f10c7fff700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f10bc0b2990 0x7f10bc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:49.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.374+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f10c00e2560 con 0x7f10d006b220 2024-05-14T17:00:49.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.375+0000 7f10c7fff700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f10bc0b2990 0x7f10bc0b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b5c0 tx=0x7f10b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:49.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.377+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10c0020020 con 0x7f10d006b220 2024-05-14T17:00:49.038 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:48 smithi099 ceph-mon[31851]: pgmap v302: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-14T17:00:49.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.630+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10d019aff0 con 0x7f10d006b220 2024-05-14T17:00:49.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.631+0000 7f10c77fe700 1 -- 172.21.15.99:0/2609758856 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7f10c00aa250 con 0x7f10d006b220 2024-05-14T17:00:49.080 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:48 smithi193 ceph-mon[35962]: pgmap v302: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-14T17:00:49.104 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:49.158 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:00:30.962944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:00:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f10bc0b2990 msgr2=0x7f10bc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:49.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f10bc0b2990 0x7f10bc0b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b5c0 tx=0x7f10b8005fd0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 msgr2=0x7f10d019f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:49.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d019f940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f10c000d980 tx=0x7f10c000dc90 comp rx=0 tx=0).stop 2024-05-14T17:00:49.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 shutdown_connections 2024-05-14T17:00:49.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f10bc0b2990 0x7f10bc0b4e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10d01095b0 0x7f10d0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10d006b220 0x7f10d019f940 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 --2- 172.21.15.99:0/2609758856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d006bbe0 0x7f10d019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:49.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.633+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 >> 172.21.15.99:0/2609758856 conn(0x7f10d00793f0 msgr2=0x7f10d01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:49.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.634+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 shutdown_connections 2024-05-14T17:00:49.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:48.634+0000 7f10d4f00700 1 -- 172.21.15.99:0/2609758856 wait complete. 2024-05-14T17:00:49.329 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:00:49.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:00:49.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:49 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2609758856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:49.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:49 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2609758856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:49 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2609758856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:49.600 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:50.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:50 smithi150 ceph-mon[36204]: pgmap v303: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-14T17:00:50.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:50 smithi099 ceph-mon[31851]: pgmap v303: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-14T17:00:50.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:50 smithi193 ceph-mon[35962]: pgmap v303: 97 pgs: 97 active+clean; 622 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-05-14T17:00:51.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 -- 172.21.15.99:0/3107234410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c1095b0 msgr2=0x7f357c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 --2- 172.21.15.99:0/3107234410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c1095b0 0x7f357c109990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3570009990 tx=0x7f3570009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 -- 172.21.15.99:0/3107234410 shutdown_connections 2024-05-14T17:00:51.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 --2- 172.21.15.99:0/3107234410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f357c06bbe0 0x7f357c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 --2- 172.21.15.99:0/3107234410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c1095b0 0x7f357c109990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 --2- 172.21.15.99:0/3107234410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f357c06b220 0x7f357c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 -- 172.21.15.99:0/3107234410 >> 172.21.15.99:0/3107234410 conn(0x7f357c0793f0 msgr2=0x7f357c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 -- 172.21.15.99:0/3107234410 shutdown_connections 2024-05-14T17:00:51.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.219+0000 7f3582db4700 1 -- 172.21.15.99:0/3107234410 wait complete. 2024-05-14T17:00:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 Processor -- start 2024-05-14T17:00:51.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 -- start start 2024-05-14T17:00:51.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 0x7f357c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:51.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f357c06bbe0 0x7f357c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:51.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f357c1095b0 0x7f357c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:51.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f357c1136c0 con 0x7f357c06b220 2024-05-14T17:00:51.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f357c113540 con 0x7f357c1095b0 2024-05-14T17:00:51.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3582db4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f357c113840 con 0x7f357c06bbe0 2024-05-14T17:00:51.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 0x7f357c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:51.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 0x7f357c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58342/0 (socket says 172.21.15.99:58342) 2024-05-14T17:00:51.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 -- 172.21.15.99:0/2326703533 learned_addr learned my addr 172.21.15.99:0/2326703533 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:51.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f357bfff700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f357c06bbe0 0x7f357c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:51.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3581351700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f357c1095b0 0x7f357c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:51.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 -- 172.21.15.99:0/2326703533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f357c06bbe0 msgr2=0x7f357c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f357c06bbe0 0x7f357c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 -- 172.21.15.99:0/2326703533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f357c1095b0 msgr2=0x7f357c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f357c1095b0 0x7f357c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 -- 172.21.15.99:0/2326703533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c19a160 con 0x7f357c06b220 2024-05-14T17:00:51.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.220+0000 7f3580b50700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 0x7f357c19f840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3570000c00 tx=0x7f35700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:51.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.221+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570027070 con 0x7f357c06b220 2024-05-14T17:00:51.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.221+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357c19a3f0 con 0x7f357c06b220 2024-05-14T17:00:51.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.221+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3570003f20 con 0x7f357c06b220 2024-05-14T17:00:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.221+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570022cf0 con 0x7f357c06b220 2024-05-14T17:00:51.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.221+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357c102440 con 0x7f357c06b220 2024-05-14T17:00:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.222+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357c10ba90 con 0x7f357c06b220 2024-05-14T17:00:51.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.223+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f357002a070 con 0x7f357c06b220 2024-05-14T17:00:51.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.224+0000 7f3579ffb700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f356c0b2990 0x7f356c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.224+0000 7f357bfff700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f356c0b2990 0x7f356c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.224+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f35700e3300 con 0x7f357c06b220 2024-05-14T17:00:51.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.224+0000 7f357bfff700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f356c0b2990 0x7f356c0b4e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f357c079ad0 tx=0x7f356800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.227+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3570013070 con 0x7f357c06b220 2024-05-14T17:00:51.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.438+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f357c19af30 con 0x7f357c06b220 2024-05-14T17:00:51.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.439+0000 7f3579ffb700 1 -- 172.21.15.99:0/2326703533 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22243 (secure 0 0 0) 0x7f35700ab100 con 0x7f357c06b220 2024-05-14T17:00:51.818 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:51.833 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":80,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:00:11.851301+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:00:51.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f356c0b2990 msgr2=0x7f356c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f356c0b2990 0x7f356c0b4e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f357c079ad0 tx=0x7f356800b410 comp rx=0 tx=0).stop 2024-05-14T17:00:51.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 msgr2=0x7f357c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 0x7f357c19f840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3570000c00 tx=0x7f35700047b0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 shutdown_connections 2024-05-14T17:00:51.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f356c0b2990 0x7f356c0b4e50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f357c1095b0 0x7f357c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357c06b220 0x7f357c19f840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 --2- 172.21.15.99:0/2326703533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f357c06bbe0 0x7f357c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 >> 172.21.15.99:0/2326703533 conn(0x7f357c0793f0 msgr2=0x7f357c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:51.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 shutdown_connections 2024-05-14T17:00:51.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:51.442+0000 7f3582db4700 1 -- 172.21.15.99:0/2326703533 wait complete. 2024-05-14T17:00:52.029 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T17:00:52.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:00:52.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:52 smithi099 ceph-mon[31851]: pgmap v304: 97 pgs: 97 active+clean; 634 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-14T17:00:52.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:52 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2326703533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:52.292 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:52 smithi150 ceph-mon[36204]: pgmap v304: 97 pgs: 97 active+clean; 634 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-14T17:00:52.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:52 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2326703533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:52 smithi193 ceph-mon[35962]: pgmap v304: 97 pgs: 97 active+clean; 634 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-14T17:00:52.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:52 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2326703533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:00:53.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.890+0000 7fe6ce36e700 1 -- 172.21.15.99:0/1654144138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c8075770 msgr2=0x7fe6c8075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:53.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.890+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/1654144138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c8075770 0x7fe6c8075b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009990 tx=0x7fe6b8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 -- 172.21.15.99:0/1654144138 shutdown_connections 2024-05-14T17:00:53.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/1654144138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c810ff40 0x7fe6c8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/1654144138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c8075770 0x7fe6c8075b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/1654144138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 0x7fe6c810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 -- 172.21.15.99:0/1654144138 >> 172.21.15.99:0/1654144138 conn(0x7fe6c807a6a0 msgr2=0x7fe6c807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:53.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 -- 172.21.15.99:0/1654144138 shutdown_connections 2024-05-14T17:00:53.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 -- 172.21.15.99:0/1654144138 wait complete. 2024-05-14T17:00:54.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 Processor -- start 2024-05-14T17:00:54.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.891+0000 7fe6ce36e700 1 -- start start 2024-05-14T17:00:54.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8075770 0x7fe6c819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:54.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 0x7fe6c819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 0x7fe6c81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:54.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8115940 con 0x7fe6c810ff40 2024-05-14T17:00:54.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81157c0 con 0x7fe6c8076090 2024-05-14T17:00:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8115ac0 con 0x7fe6c8075770 2024-05-14T17:00:54.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 0x7fe6c81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 0x7fe6c81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58362/0 (socket says 172.21.15.99:58362) 2024-05-14T17:00:54.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 -- 172.21.15.99:0/3805429061 learned_addr learned my addr 172.21.15.99:0/3805429061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:54.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6c7fff700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8075770 0x7fe6c819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6bffff700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 0x7fe6c819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 -- 172.21.15.99:0/3805429061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8075770 msgr2=0x7fe6c819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8075770 0x7fe6c819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 -- 172.21.15.99:0/3805429061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 msgr2=0x7fe6c819c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:54.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 0x7fe6c819c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 -- 172.21.15.99:0/3805429061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c81a0d90 con 0x7fe6c810ff40 2024-05-14T17:00:54.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6bffff700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 0x7fe6c819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:54.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6c7fff700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8075770 0x7fe6c819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:54.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6cc90b700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 0x7fe6c81a0650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b970 tx=0x7fe6c000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:54.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0019070 con 0x7fe6c810ff40 2024-05-14T17:00:54.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c0014440 con 0x7fe6c810ff40 2024-05-14T17:00:54.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.892+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c81a1080 con 0x7fe6c810ff40 2024-05-14T17:00:54.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.893+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000ce60 con 0x7fe6c810ff40 2024-05-14T17:00:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.893+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c81a8900 con 0x7fe6c810ff40 2024-05-14T17:00:54.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.894+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c806a230 con 0x7fe6c810ff40 2024-05-14T17:00:54.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.895+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe6c001c070 con 0x7fe6c810ff40 2024-05-14T17:00:54.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.895+0000 7fe6c5ffb700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe6b40b2990 0x7fe6b40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:54.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.895+0000 7fe6c7fff700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe6b40b2990 0x7fe6b40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.895+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe6c00e1490 con 0x7fe6c810ff40 2024-05-14T17:00:54.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.896+0000 7fe6c7fff700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe6b40b2990 0x7fe6b40b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8006010 tx=0x7fe6b800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:54.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:53.900+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6c00a9310 con 0x7fe6c810ff40 2024-05-14T17:00:54.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:54 smithi193 ceph-mon[35962]: pgmap v305: 97 pgs: 97 active+clean; 634 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-14T17:00:55.065 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:54 smithi150 ceph-mon[36204]: pgmap v305: 97 pgs: 97 active+clean; 634 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-14T17:00:55.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.099+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6c8063980 con 0x7fe6b40b2990 2024-05-14T17:00:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.106+0000 7fe6c5ffb700 1 -- 172.21.15.99:0/3805429061 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258687 (secure 0 0 0) 0x7fe6c8063980 con 0x7fe6b40b2990 2024-05-14T17:00:55.168 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:55.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe6b40b2990 msgr2=0x7fe6b40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:55.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe6b40b2990 0x7fe6b40b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8006010 tx=0x7fe6b800b540 comp rx=0 tx=0).stop 2024-05-14T17:00:55.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 msgr2=0x7fe6c81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:55.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 0x7fe6c81a0650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b970 tx=0x7fe6c000bd30 comp rx=0 tx=0).stop 2024-05-14T17:00:55.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 shutdown_connections 2024-05-14T17:00:55.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe6b40b2990 0x7fe6b40b4e50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6c810ff40 0x7fe6c81a0650 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8075770 0x7fe6c819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 --2- 172.21.15.99:0/3805429061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c8076090 0x7fe6c819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 >> 172.21.15.99:0/3805429061 conn(0x7fe6c807a6a0 msgr2=0x7fe6c81006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:55.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 shutdown_connections 2024-05-14T17:00:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:54.109+0000 7fe6ce36e700 1 -- 172.21.15.99:0/3805429061 wait complete. 2024-05-14T17:00:55.623 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:00:55.630 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:54 smithi099 ceph-mon[31851]: pgmap v305: 97 pgs: 97 active+clean; 634 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-14T17:00:55.637 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-05-14T17:00:52.467396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649555,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":235,"num_read_kb":209,"num_write":283,"num_write_kb":2254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365504,"kb_used_data":12192,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576320,"statfs":{"total":1151940427776,"available":1151566151680,"internally_reserved":0,"allocated":12484608,"data_stored":6930833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32739,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":21,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067089+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:00:12.067089+0000","last_peered":"2024-05-14T17:00:12.067089+0000","last_clean":"2024-05-14T17:00:12.067089+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:00:12.067089+0000","last_undegraded":"2024-05-14T17:00:12.067089+0000","last_fullsized":"2024-05-14T17:00:12.067089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066684+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:00:12.066684+0000","last_peered":"2024-05-14T17:00:12.066684+0000","last_clean":"2024-05-14T17:00:12.066684+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:00:12.066684+0000","last_undegraded":"2024-05-14T17:00:12.066684+0000","last_fullsized":"2024-05-14T17:00:12.066684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066609+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:00:12.066609+0000","last_peered":"2024-05-14T17:00:12.066609+0000","last_clean":"2024-05-14T17:00:12.066609+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:00:12.066609+0000","last_undegraded":"2024-05-14T17:00:12.066609+0000","last_fullsized":"2024-05-14T17:00:12.066609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063154+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:00:12.063154+0000","last_peered":"2024-05-14T17:00:12.063154+0000","last_clean":"2024-05-14T17:00:12.063154+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:00:12.063154+0000","last_undegraded":"2024-05-14T17:00:12.063154+0000","last_fullsized":"2024-05-14T17:00:12.063154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063448+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:00:12.063448+0000","last_peered":"2024-05-14T17:00:12.063448+0000","last_clean":"2024-05-14T17:00:12.063448+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:00:12.063448+0000","last_undegraded":"2024-05-14T17:00:12.063448+0000","last_fullsized":"2024-05-14T17:00:12.063448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062788+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:00:12.062788+0000","last_peered":"2024-05-14T17:00:12.062788+0000","last_clean":"2024-05-14T17:00:12.062788+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:00:12.062788+0000","last_undegraded":"2024-05-14T17:00:12.062788+0000","last_fullsized":"2024-05-14T17:00:12.062788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066581+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:00:12.066581+0000","last_peered":"2024-05-14T17:00:12.066581+0000","last_clean":"2024-05-14T17:00:12.066581+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:00:12.066581+0000","last_undegraded":"2024-05-14T17:00:12.066581+0000","last_fullsized":"2024-05-14T17:00:12.066581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055561+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:00:12.055561+0000","last_peered":"2024-05-14T17:00:12.055561+0000","last_clean":"2024-05-14T17:00:12.055561+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:00:12.055561+0000","last_undegraded":"2024-05-14T17:00:12.055561+0000","last_fullsized":"2024-05-14T17:00:12.055561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058946+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:00:12.058946+0000","last_peered":"2024-05-14T17:00:12.058946+0000","last_clean":"2024-05-14T17:00:12.058946+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:00:12.058946+0000","last_undegraded":"2024-05-14T17:00:12.058946+0000","last_fullsized":"2024-05-14T17:00:12.058946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063879+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:00:12.063879+0000","last_peered":"2024-05-14T17:00:12.063879+0000","last_clean":"2024-05-14T17:00:12.063879+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:00:12.063879+0000","last_undegraded":"2024-05-14T17:00:12.063879+0000","last_fullsized":"2024-05-14T17:00:12.063879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067726+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:00:12.067726+0000","last_peered":"2024-05-14T17:00:12.067726+0000","last_clean":"2024-05-14T17:00:12.067726+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:00:12.067726+0000","last_undegraded":"2024-05-14T17:00:12.067726+0000","last_fullsized":"2024-05-14T17:00:12.067726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062996+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:00:12.062996+0000","last_peered":"2024-05-14T17:00:12.062996+0000","last_clean":"2024-05-14T17:00:12.062996+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:00:12.062996+0000","last_undegraded":"2024-05-14T17:00:12.062996+0000","last_fullsized":"2024-05-14T17:00:12.062996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063365+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:00:12.063365+0000","last_peered":"2024-05-14T17:00:12.063365+0000","last_clean":"2024-05-14T17:00:12.063365+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:00:12.063365+0000","last_undegraded":"2024-05-14T17:00:12.063365+0000","last_fullsized":"2024-05-14T17:00:12.063365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.065791+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:00:12.065791+0000","last_peered":"2024-05-14T17:00:12.065791+0000","last_clean":"2024-05-14T17:00:12.065791+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:00:12.065791+0000","last_undegraded":"2024-05-14T17:00:12.065791+0000","last_fullsized":"2024-05-14T17:00:12.065791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066460+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:00:12.066460+0000","last_peered":"2024-05-14T17:00:12.066460+0000","last_clean":"2024-05-14T17:00:12.066460+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:00:12.066460+0000","last_undegraded":"2024-05-14T17:00:12.066460+0000","last_fullsized":"2024-05-14T17:00:12.066460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058251+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:00:12.058251+0000","last_peered":"2024-05-14T17:00:12.058251+0000","last_clean":"2024-05-14T17:00:12.058251+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:00:12.058251+0000","last_undegraded":"2024-05-14T17:00:12.058251+0000","last_fullsized":"2024-05-14T17:00:12.058251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067247+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:00:12.067247+0000","last_peered":"2024-05-14T17:00:12.067247+0000","last_clean":"2024-05-14T17:00:12.067247+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:00:12.067247+0000","last_undegraded":"2024-05-14T17:00:12.067247+0000","last_fullsized":"2024-05-14T17:00:12.067247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062756+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:00:12.062756+0000","last_peered":"2024-05-14T17:00:12.062756+0000","last_clean":"2024-05-14T17:00:12.062756+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:00:12.062756+0000","last_undegraded":"2024-05-14T17:00:12.062756+0000","last_fullsized":"2024-05-14T17:00:12.062756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.064067+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:00:12.064067+0000","last_peered":"2024-05-14T17:00:12.064067+0000","last_clean":"2024-05-14T17:00:12.064067+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:00:12.064067+0000","last_undegraded":"2024-05-14T17:00:12.064067+0000","last_fullsized":"2024-05-14T17:00:12.064067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125952+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:00:26.125952+0000","last_peered":"2024-05-14T17:00:26.125952+0000","last_clean":"2024-05-14T17:00:26.125952+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:00:26.125952+0000","last_undegraded":"2024-05-14T17:00:26.125952+0000","last_fullsized":"2024-05-14T17:00:26.125952+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055761+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:00:12.055761+0000","last_peered":"2024-05-14T17:00:12.055761+0000","last_clean":"2024-05-14T17:00:12.055761+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:00:12.055761+0000","last_undegraded":"2024-05-14T17:00:12.055761+0000","last_fullsized":"2024-05-14T17:00:12.055761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.065500+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:00:12.065500+0000","last_peered":"2024-05-14T17:00:12.065500+0000","last_clean":"2024-05-14T17:00:12.065500+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:00:12.065500+0000","last_undegraded":"2024-05-14T17:00:12.065500+0000","last_fullsized":"2024-05-14T17:00:12.065500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066866+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:00:12.066866+0000","last_peered":"2024-05-14T17:00:12.066866+0000","last_clean":"2024-05-14T17:00:12.066866+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:00:12.066866+0000","last_undegraded":"2024-05-14T17:00:12.066866+0000","last_fullsized":"2024-05-14T17:00:12.066866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.124887+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:00:26.124887+0000","last_peered":"2024-05-14T17:00:26.124887+0000","last_clean":"2024-05-14T17:00:26.124887+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:00:26.124887+0000","last_undegraded":"2024-05-14T17:00:26.124887+0000","last_fullsized":"2024-05-14T17:00:26.124887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058480+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:00:12.058480+0000","last_peered":"2024-05-14T17:00:12.058480+0000","last_clean":"2024-05-14T17:00:12.058480+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:00:12.058480+0000","last_undegraded":"2024-05-14T17:00:12.058480+0000","last_fullsized":"2024-05-14T17:00:12.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.070478+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:00:12.070478+0000","last_peered":"2024-05-14T17:00:12.070478+0000","last_clean":"2024-05-14T17:00:12.070478+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:00:12.070478+0000","last_undegraded":"2024-05-14T17:00:12.070478+0000","last_fullsized":"2024-05-14T17:00:12.070478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056810+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:00:12.056810+0000","last_peered":"2024-05-14T17:00:12.056810+0000","last_clean":"2024-05-14T17:00:12.056810+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:00:12.056810+0000","last_undegraded":"2024-05-14T17:00:12.056810+0000","last_fullsized":"2024-05-14T17:00:12.056810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:47.168827+0000","last_change":"2024-05-14T17:00:07.040846+0000","last_active":"2024-05-14T17:00:47.168827+0000","last_peered":"2024-05-14T17:00:47.168827+0000","last_clean":"2024-05-14T17:00:47.168827+0000","last_became_active":"2024-05-14T17:00:07.040449+0000","last_became_peered":"2024-05-14T17:00:07.040449+0000","last_unstale":"2024-05-14T17:00:47.168827+0000","last_undegraded":"2024-05-14T17:00:47.168827+0000","last_fullsized":"2024-05-14T17:00:47.168827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:52.192689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063647+0000","last_change":"2024-05-14T17:00:10.054669+0000","last_active":"2024-05-14T17:00:12.063647+0000","last_peered":"2024-05-14T17:00:12.063647+0000","last_clean":"2024-05-14T17:00:12.063647+0000","last_became_active":"2024-05-14T17:00:10.054298+0000","last_became_peered":"2024-05-14T17:00:10.054298+0000","last_unstale":"2024-05-14T17:00:12.063647+0000","last_undegraded":"2024-05-14T17:00:12.063647+0000","last_fullsized":"2024-05-14T17:00:12.063647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:35:54.172686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.124142+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:00:26.124142+0000","last_peered":"2024-05-14T17:00:26.124142+0000","last_clean":"2024-05-14T17:00:26.124142+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:00:26.124142+0000","last_undegraded":"2024-05-14T17:00:26.124142+0000","last_fullsized":"2024-05-14T17:00:26.124142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056898+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:00:12.056898+0000","last_peered":"2024-05-14T17:00:12.056898+0000","last_clean":"2024-05-14T17:00:12.056898+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:00:12.056898+0000","last_undegraded":"2024-05-14T17:00:12.056898+0000","last_fullsized":"2024-05-14T17:00:12.056898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.124887+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:00:26.124887+0000","last_peered":"2024-05-14T17:00:26.124887+0000","last_clean":"2024-05-14T17:00:26.124887+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:00:26.124887+0000","last_undegraded":"2024-05-14T17:00:26.124887+0000","last_fullsized":"2024-05-14T17:00:26.124887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055644+0000","last_change":"2024-05-14T17:00:10.058307+0000","last_active":"2024-05-14T17:00:12.055644+0000","last_peered":"2024-05-14T17:00:12.055644+0000","last_clean":"2024-05-14T17:00:12.055644+0000","last_became_active":"2024-05-14T17:00:10.058016+0000","last_became_peered":"2024-05-14T17:00:10.058016+0000","last_unstale":"2024-05-14T17:00:12.055644+0000","last_undegraded":"2024-05-14T17:00:12.055644+0000","last_fullsized":"2024-05-14T17:00:12.055644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:51.000609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056328+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:00:12.056328+0000","last_peered":"2024-05-14T17:00:12.056328+0000","last_clean":"2024-05-14T17:00:12.056328+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:00:12.056328+0000","last_undegraded":"2024-05-14T17:00:12.056328+0000","last_fullsized":"2024-05-14T17:00:12.056328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.061983+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:00:12.061983+0000","last_peered":"2024-05-14T17:00:12.061983+0000","last_clean":"2024-05-14T17:00:12.061983+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:00:12.061983+0000","last_undegraded":"2024-05-14T17:00:12.061983+0000","last_fullsized":"2024-05-14T17:00:12.061983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:47.168120+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:00:47.168120+0000","last_peered":"2024-05-14T17:00:47.168120+0000","last_clean":"2024-05-14T17:00:47.168120+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:00:47.168120+0000","last_undegraded":"2024-05-14T17:00:47.168120+0000","last_fullsized":"2024-05-14T17:00:47.168120+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058894+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:00:12.058894+0000","last_peered":"2024-05-14T17:00:12.058894+0000","last_clean":"2024-05-14T17:00:12.058894+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:00:12.058894+0000","last_undegraded":"2024-05-14T17:00:12.058894+0000","last_fullsized":"2024-05-14T17:00:12.058894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119686+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:00:25.119686+0000","last_peered":"2024-05-14T17:00:25.119686+0000","last_clean":"2024-05-14T17:00:25.119686+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:00:25.119686+0000","last_undegraded":"2024-05-14T17:00:25.119686+0000","last_fullsized":"2024-05-14T17:00:25.119686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066819+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:00:12.066819+0000","last_peered":"2024-05-14T17:00:12.066819+0000","last_clean":"2024-05-14T17:00:12.066819+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:00:12.066819+0000","last_undegraded":"2024-05-14T17:00:12.066819+0000","last_fullsized":"2024-05-14T17:00:12.066819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.118724+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:00:25.118724+0000","last_peered":"2024-05-14T17:00:25.118724+0000","last_clean":"2024-05-14T17:00:25.118724+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:00:25.118724+0000","last_undegraded":"2024-05-14T17:00:25.118724+0000","last_fullsized":"2024-05-14T17:00:25.118724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058608+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:00:12.058608+0000","last_peered":"2024-05-14T17:00:12.058608+0000","last_clean":"2024-05-14T17:00:12.058608+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:00:12.058608+0000","last_undegraded":"2024-05-14T17:00:12.058608+0000","last_fullsized":"2024-05-14T17:00:12.058608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'25","reported_seq":96,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:48.170172+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:00:48.170172+0000","last_peered":"2024-05-14T17:00:48.170172+0000","last_clean":"2024-05-14T17:00:48.170172+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:00:48.170172+0000","last_undegraded":"2024-05-14T17:00:48.170172+0000","last_fullsized":"2024-05-14T17:00:48.170172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54952,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":77,"num_write":26,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063582+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:00:12.063582+0000","last_peered":"2024-05-14T17:00:12.063582+0000","last_clean":"2024-05-14T17:00:12.063582+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:00:12.063582+0000","last_undegraded":"2024-05-14T17:00:12.063582+0000","last_fullsized":"2024-05-14T17:00:12.063582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125102+0000","last_change":"2024-05-14T17:00:07.038589+0000","last_active":"2024-05-14T17:00:26.125102+0000","last_peered":"2024-05-14T17:00:26.125102+0000","last_clean":"2024-05-14T17:00:26.125102+0000","last_became_active":"2024-05-14T17:00:07.037870+0000","last_became_peered":"2024-05-14T17:00:07.037870+0000","last_unstale":"2024-05-14T17:00:26.125102+0000","last_undegraded":"2024-05-14T17:00:26.125102+0000","last_fullsized":"2024-05-14T17:00:26.125102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:49:21.471467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062915+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:00:12.062915+0000","last_peered":"2024-05-14T17:00:12.062915+0000","last_clean":"2024-05-14T17:00:12.062915+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:00:12.062915+0000","last_undegraded":"2024-05-14T17:00:12.062915+0000","last_fullsized":"2024-05-14T17:00:12.062915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119040+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:00:25.119040+0000","last_peered":"2024-05-14T17:00:25.119040+0000","last_clean":"2024-05-14T17:00:25.119040+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:00:25.119040+0000","last_undegraded":"2024-05-14T17:00:25.119040+0000","last_fullsized":"2024-05-14T17:00:25.119040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067254+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:00:12.067254+0000","last_peered":"2024-05-14T17:00:12.067254+0000","last_clean":"2024-05-14T17:00:12.067254+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:00:12.067254+0000","last_undegraded":"2024-05-14T17:00:12.067254+0000","last_fullsized":"2024-05-14T17:00:12.067254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:47.164067+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:00:47.164067+0000","last_peered":"2024-05-14T17:00:47.164067+0000","last_clean":"2024-05-14T17:00:47.164067+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:00:47.164067+0000","last_undegraded":"2024-05-14T17:00:47.164067+0000","last_fullsized":"2024-05-14T17:00:47.164067+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066827+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:00:12.066827+0000","last_peered":"2024-05-14T17:00:12.066827+0000","last_clean":"2024-05-14T17:00:12.066827+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:00:12.066827+0000","last_undegraded":"2024-05-14T17:00:12.066827+0000","last_fullsized":"2024-05-14T17:00:12.066827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.127428+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:00:26.127428+0000","last_peered":"2024-05-14T17:00:26.127428+0000","last_clean":"2024-05-14T17:00:26.127428+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:00:26.127428+0000","last_undegraded":"2024-05-14T17:00:26.127428+0000","last_fullsized":"2024-05-14T17:00:26.127428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056397+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:00:12.056397+0000","last_peered":"2024-05-14T17:00:12.056397+0000","last_clean":"2024-05-14T17:00:12.056397+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:00:12.056397+0000","last_undegraded":"2024-05-14T17:00:12.056397+0000","last_fullsized":"2024-05-14T17:00:12.056397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:28.121726+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:00:28.121726+0000","last_peered":"2024-05-14T17:00:28.121726+0000","last_clean":"2024-05-14T17:00:28.121726+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:00:28.121726+0000","last_undegraded":"2024-05-14T17:00:28.121726+0000","last_fullsized":"2024-05-14T17:00:28.121726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066947+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:00:12.066947+0000","last_peered":"2024-05-14T17:00:12.066947+0000","last_clean":"2024-05-14T17:00:12.066947+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:00:12.066947+0000","last_undegraded":"2024-05-14T17:00:12.066947+0000","last_fullsized":"2024-05-14T17:00:12.066947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.061816+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:00:12.061816+0000","last_peered":"2024-05-14T17:00:12.061816+0000","last_clean":"2024-05-14T17:00:12.061816+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:00:12.061816+0000","last_undegraded":"2024-05-14T17:00:12.061816+0000","last_fullsized":"2024-05-14T17:00:12.061816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:13.923463+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:00:13.923463+0000","last_peered":"2024-05-14T17:00:13.923463+0000","last_clean":"2024-05-14T17:00:13.923463+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:00:13.923463+0000","last_undegraded":"2024-05-14T17:00:13.923463+0000","last_fullsized":"2024-05-14T17:00:13.923463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.068021+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:00:12.068021+0000","last_peered":"2024-05-14T17:00:12.068021+0000","last_clean":"2024-05-14T17:00:12.068021+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:00:12.068021+0000","last_undegraded":"2024-05-14T17:00:12.068021+0000","last_fullsized":"2024-05-14T17:00:12.068021+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062915+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:00:12.062915+0000","last_peered":"2024-05-14T17:00:12.062915+0000","last_clean":"2024-05-14T17:00:12.062915+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:00:12.062915+0000","last_undegraded":"2024-05-14T17:00:12.062915+0000","last_fullsized":"2024-05-14T17:00:12.062915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056628+0000","last_change":"2024-05-14T17:00:10.060967+0000","last_active":"2024-05-14T17:00:12.056628+0000","last_peered":"2024-05-14T17:00:12.056628+0000","last_clean":"2024-05-14T17:00:12.056628+0000","last_became_active":"2024-05-14T17:00:10.060392+0000","last_became_peered":"2024-05-14T17:00:10.060392+0000","last_unstale":"2024-05-14T17:00:12.056628+0000","last_undegraded":"2024-05-14T17:00:12.056628+0000","last_fullsized":"2024-05-14T17:00:12.056628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:25:13.345462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":49,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:49.166212+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:00:49.166212+0000","last_peered":"2024-05-14T17:00:49.166212+0000","last_clean":"2024-05-14T17:00:49.166212+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:00:49.166212+0000","last_undegraded":"2024-05-14T17:00:49.166212+0000","last_fullsized":"2024-05-14T17:00:49.166212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056070+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:00:12.056070+0000","last_peered":"2024-05-14T17:00:12.056070+0000","last_clean":"2024-05-14T17:00:12.056070+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:00:12.056070+0000","last_undegraded":"2024-05-14T17:00:12.056070+0000","last_fullsized":"2024-05-14T17:00:12.056070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:28.132394+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:00:28.132394+0000","last_peered":"2024-05-14T17:00:28.132394+0000","last_clean":"2024-05-14T17:00:28.132394+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:00:28.132394+0000","last_undegraded":"2024-05-14T17:00:28.132394+0000","last_fullsized":"2024-05-14T17:00:28.132394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067036+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:00:12.067036+0000","last_peered":"2024-05-14T17:00:12.067036+0000","last_clean":"2024-05-14T17:00:12.067036+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:00:12.067036+0000","last_undegraded":"2024-05-14T17:00:12.067036+0000","last_fullsized":"2024-05-14T17:00:12.067036+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062530+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:00:12.062530+0000","last_peered":"2024-05-14T17:00:12.062530+0000","last_clean":"2024-05-14T17:00:12.062530+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:00:12.062530+0000","last_undegraded":"2024-05-14T17:00:12.062530+0000","last_fullsized":"2024-05-14T17:00:12.062530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125897+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:00:26.125897+0000","last_peered":"2024-05-14T17:00:26.125897+0000","last_clean":"2024-05-14T17:00:26.125897+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:00:26.125897+0000","last_undegraded":"2024-05-14T17:00:26.125897+0000","last_fullsized":"2024-05-14T17:00:26.125897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.117959+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:00:25.117959+0000","last_peered":"2024-05-14T17:00:25.117959+0000","last_clean":"2024-05-14T17:00:25.117959+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:00:25.117959+0000","last_undegraded":"2024-05-14T17:00:25.117959+0000","last_fullsized":"2024-05-14T17:00:25.117959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063284+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:00:12.063284+0000","last_peered":"2024-05-14T17:00:12.063284+0000","last_clean":"2024-05-14T17:00:12.063284+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:00:12.063284+0000","last_undegraded":"2024-05-14T17:00:12.063284+0000","last_fullsized":"2024-05-14T17:00:12.063284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:45.064907+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:00:45.064907+0000","last_peered":"2024-05-14T17:00:45.064907+0000","last_clean":"2024-05-14T17:00:45.064907+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:00:45.064907+0000","last_undegraded":"2024-05-14T17:00:45.064907+0000","last_fullsized":"2024-05-14T17:00:45.064907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056049+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:00:12.056049+0000","last_peered":"2024-05-14T17:00:12.056049+0000","last_clean":"2024-05-14T17:00:12.056049+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:00:12.056049+0000","last_undegraded":"2024-05-14T17:00:12.056049+0000","last_fullsized":"2024-05-14T17:00:12.056049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.118018+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:00:25.118018+0000","last_peered":"2024-05-14T17:00:25.118018+0000","last_clean":"2024-05-14T17:00:25.118018+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:00:25.118018+0000","last_undegraded":"2024-05-14T17:00:25.118018+0000","last_fullsized":"2024-05-14T17:00:25.118018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066868+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:00:12.066868+0000","last_peered":"2024-05-14T17:00:12.066868+0000","last_clean":"2024-05-14T17:00:12.066868+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:00:12.066868+0000","last_undegraded":"2024-05-14T17:00:12.066868+0000","last_fullsized":"2024-05-14T17:00:12.066868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119381+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:00:25.119381+0000","last_peered":"2024-05-14T17:00:25.119381+0000","last_clean":"2024-05-14T17:00:25.119381+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:00:25.119381+0000","last_undegraded":"2024-05-14T17:00:25.119381+0000","last_fullsized":"2024-05-14T17:00:25.119381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.064843+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:00:12.064843+0000","last_peered":"2024-05-14T17:00:12.064843+0000","last_clean":"2024-05-14T17:00:12.064843+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:00:12.064843+0000","last_undegraded":"2024-05-14T17:00:12.064843+0000","last_fullsized":"2024-05-14T17:00:12.064843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.117355+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:00:25.117355+0000","last_peered":"2024-05-14T17:00:25.117355+0000","last_clean":"2024-05-14T17:00:25.117355+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:00:25.117355+0000","last_undegraded":"2024-05-14T17:00:25.117355+0000","last_fullsized":"2024-05-14T17:00:25.117355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067409+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:00:12.067409+0000","last_peered":"2024-05-14T17:00:12.067409+0000","last_clean":"2024-05-14T17:00:12.067409+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:00:12.067409+0000","last_undegraded":"2024-05-14T17:00:12.067409+0000","last_fullsized":"2024-05-14T17:00:12.067409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063948+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:00:12.063948+0000","last_peered":"2024-05-14T17:00:12.063948+0000","last_clean":"2024-05-14T17:00:12.063948+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:00:12.063948+0000","last_undegraded":"2024-05-14T17:00:12.063948+0000","last_fullsized":"2024-05-14T17:00:12.063948+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055792+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:00:12.055792+0000","last_peered":"2024-05-14T17:00:12.055792+0000","last_clean":"2024-05-14T17:00:12.055792+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:00:12.055792+0000","last_undegraded":"2024-05-14T17:00:12.055792+0000","last_fullsized":"2024-05-14T17:00:12.055792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119414+0000","last_change":"2024-05-14T17:00:07.043988+0000","last_active":"2024-05-14T17:00:25.119414+0000","last_peered":"2024-05-14T17:00:25.119414+0000","last_clean":"2024-05-14T17:00:25.119414+0000","last_became_active":"2024-05-14T17:00:07.043620+0000","last_became_peered":"2024-05-14T17:00:07.043620+0000","last_unstale":"2024-05-14T17:00:25.119414+0000","last_undegraded":"2024-05-14T17:00:25.119414+0000","last_fullsized":"2024-05-14T17:00:25.119414+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:03.650546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067758+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:00:12.067758+0000","last_peered":"2024-05-14T17:00:12.067758+0000","last_clean":"2024-05-14T17:00:12.067758+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:00:12.067758+0000","last_undegraded":"2024-05-14T17:00:12.067758+0000","last_fullsized":"2024-05-14T17:00:12.067758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125373+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:00:26.125373+0000","last_peered":"2024-05-14T17:00:26.125373+0000","last_clean":"2024-05-14T17:00:26.125373+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:00:26.125373+0000","last_undegraded":"2024-05-14T17:00:26.125373+0000","last_fullsized":"2024-05-14T17:00:26.125373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067995+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:00:12.067995+0000","last_peered":"2024-05-14T17:00:12.067995+0000","last_clean":"2024-05-14T17:00:12.067995+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:00:12.067995+0000","last_undegraded":"2024-05-14T17:00:12.067995+0000","last_fullsized":"2024-05-14T17:00:12.067995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.069089+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:00:12.069089+0000","last_peered":"2024-05-14T17:00:12.069089+0000","last_clean":"2024-05-14T17:00:12.069089+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:00:12.069089+0000","last_undegraded":"2024-05-14T17:00:12.069089+0000","last_fullsized":"2024-05-14T17:00:12.069089+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067634+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:00:12.067634+0000","last_peered":"2024-05-14T17:00:12.067634+0000","last_clean":"2024-05-14T17:00:12.067634+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:00:12.067634+0000","last_undegraded":"2024-05-14T17:00:12.067634+0000","last_fullsized":"2024-05-14T17:00:12.067634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.071718+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:00:12.071718+0000","last_peered":"2024-05-14T17:00:12.071718+0000","last_clean":"2024-05-14T17:00:12.071718+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:00:12.071718+0000","last_undegraded":"2024-05-14T17:00:12.071718+0000","last_fullsized":"2024-05-14T17:00:12.071718+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067340+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:00:12.067340+0000","last_peered":"2024-05-14T17:00:12.067340+0000","last_clean":"2024-05-14T17:00:12.067340+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:00:12.067340+0000","last_undegraded":"2024-05-14T17:00:12.067340+0000","last_fullsized":"2024-05-14T17:00:12.067340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067643+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:00:12.067643+0000","last_peered":"2024-05-14T17:00:12.067643+0000","last_clean":"2024-05-14T17:00:12.067643+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:00:12.067643+0000","last_undegraded":"2024-05-14T17:00:12.067643+0000","last_fullsized":"2024-05-14T17:00:12.067643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063307+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:00:12.063307+0000","last_peered":"2024-05-14T17:00:12.063307+0000","last_clean":"2024-05-14T17:00:12.063307+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:00:12.063307+0000","last_undegraded":"2024-05-14T17:00:12.063307+0000","last_fullsized":"2024-05-14T17:00:12.063307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063682+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:00:12.063682+0000","last_peered":"2024-05-14T17:00:12.063682+0000","last_clean":"2024-05-14T17:00:12.063682+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:00:12.063682+0000","last_undegraded":"2024-05-14T17:00:12.063682+0000","last_fullsized":"2024-05-14T17:00:12.063682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066199+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:00:12.066199+0000","last_peered":"2024-05-14T17:00:12.066199+0000","last_clean":"2024-05-14T17:00:12.066199+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:00:12.066199+0000","last_undegraded":"2024-05-14T17:00:12.066199+0000","last_fullsized":"2024-05-14T17:00:12.066199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063441+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:00:12.063441+0000","last_peered":"2024-05-14T17:00:12.063441+0000","last_clean":"2024-05-14T17:00:12.063441+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:00:12.063441+0000","last_undegraded":"2024-05-14T17:00:12.063441+0000","last_fullsized":"2024-05-14T17:00:12.063441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066919+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:00:12.066919+0000","last_peered":"2024-05-14T17:00:12.066919+0000","last_clean":"2024-05-14T17:00:12.066919+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:00:12.066919+0000","last_undegraded":"2024-05-14T17:00:12.066919+0000","last_fullsized":"2024-05-14T17:00:12.066919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.064600+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:00:12.064600+0000","last_peered":"2024-05-14T17:00:12.064600+0000","last_clean":"2024-05-14T17:00:12.064600+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:00:12.064600+0000","last_undegraded":"2024-05-14T17:00:12.064600+0000","last_fullsized":"2024-05-14T17:00:12.064600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055821+0000","last_change":"2024-05-14T17:00:10.068405+0000","last_active":"2024-05-14T17:00:12.055821+0000","last_peered":"2024-05-14T17:00:12.055821+0000","last_clean":"2024-05-14T17:00:12.055821+0000","last_became_active":"2024-05-14T17:00:10.068075+0000","last_became_peered":"2024-05-14T17:00:10.068075+0000","last_unstale":"2024-05-14T17:00:12.055821+0000","last_undegraded":"2024-05-14T17:00:12.055821+0000","last_fullsized":"2024-05-14T17:00:12.055821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:49.394248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056246+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:00:12.056246+0000","last_peered":"2024-05-14T17:00:12.056246+0000","last_clean":"2024-05-14T17:00:12.056246+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:00:12.056246+0000","last_undegraded":"2024-05-14T17:00:12.056246+0000","last_fullsized":"2024-05-14T17:00:12.056246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055597+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:00:12.055597+0000","last_peered":"2024-05-14T17:00:12.055597+0000","last_clean":"2024-05-14T17:00:12.055597+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:00:12.055597+0000","last_undegraded":"2024-05-14T17:00:12.055597+0000","last_fullsized":"2024-05-14T17:00:12.055597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058488+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:00:12.058488+0000","last_peered":"2024-05-14T17:00:12.058488+0000","last_clean":"2024-05-14T17:00:12.058488+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:00:12.058488+0000","last_undegraded":"2024-05-14T17:00:12.058488+0000","last_fullsized":"2024-05-14T17:00:12.058488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063509+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:00:12.063509+0000","last_peered":"2024-05-14T17:00:12.063509+0000","last_clean":"2024-05-14T17:00:12.063509+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:00:12.063509+0000","last_undegraded":"2024-05-14T17:00:12.063509+0000","last_fullsized":"2024-05-14T17:00:12.063509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066997+0000","last_change":"2024-05-14T17:00:10.059170+0000","last_active":"2024-05-14T17:00:12.066997+0000","last_peered":"2024-05-14T17:00:12.066997+0000","last_clean":"2024-05-14T17:00:12.066997+0000","last_became_active":"2024-05-14T17:00:10.057455+0000","last_became_peered":"2024-05-14T17:00:10.057455+0000","last_unstale":"2024-05-14T17:00:12.066997+0000","last_undegraded":"2024-05-14T17:00:12.066997+0000","last_fullsized":"2024-05-14T17:00:12.066997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:45.202488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58953,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":154,"num_read_kb":140,"num_write":158,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":179135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645336,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":415573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1062006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":415729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":1007659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":415198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":471081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":884736,"data_stored":416469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":472822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":415509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":415635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1482752,"data_stored":1007343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:00:55.667 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:00:55.676 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:55 smithi099 ceph-mon[31851]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:00:55.702 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:55 smithi193 ceph-mon[35962]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:00:55.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:55 smithi150 ceph-mon[36204]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:00:55.955 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:56.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:56 smithi099 ceph-mon[31851]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T17:00:56.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:56 smithi193 ceph-mon[35962]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T17:00:56.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:56 smithi150 ceph-mon[36204]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-14T17:00:57.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 -- 172.21.15.99:0/1479162014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc181041d0 msgr2=0x7efc181081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 --2- 172.21.15.99:0/1479162014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc181041d0 0x7efc181081e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0099c0 tx=0x7efc0c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 -- 172.21.15.99:0/1479162014 shutdown_connections 2024-05-14T17:00:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 --2- 172.21.15.99:0/1479162014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc18108910 0x7efc18110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 --2- 172.21.15.99:0/1479162014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc18103820 0x7efc18103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 --2- 172.21.15.99:0/1479162014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc181041d0 0x7efc181081e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 -- 172.21.15.99:0/1479162014 >> 172.21.15.99:0/1479162014 conn(0x7efc1807a6b0 msgr2=0x7efc1807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:57.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 -- 172.21.15.99:0/1479162014 shutdown_connections 2024-05-14T17:00:57.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.647+0000 7efc1d883700 1 -- 172.21.15.99:0/1479162014 wait complete. 2024-05-14T17:00:57.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 Processor -- start 2024-05-14T17:00:57.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 -- start start 2024-05-14T17:00:57.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 0x7efc1819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc181041d0 0x7efc1819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc18108910 0x7efc18199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc18113750 con 0x7efc18103820 2024-05-14T17:00:57.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc181135d0 con 0x7efc181041d0 2024-05-14T17:00:57.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.648+0000 7efc1d883700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc181138d0 con 0x7efc18108910 2024-05-14T17:00:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 0x7efc1819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 0x7efc1819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44712/0 (socket says 172.21.15.99:44712) 2024-05-14T17:00:57.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 -- 172.21.15.99:0/3411599346 learned_addr learned my addr 172.21.15.99:0/3411599346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:00:57.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc167fc700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc181041d0 0x7efc1819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc177fe700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc18108910 0x7efc18199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 -- 172.21.15.99:0/3411599346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc18108910 msgr2=0x7efc18199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc18108910 0x7efc18199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 -- 172.21.15.99:0/3411599346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc181041d0 msgr2=0x7efc1819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc181041d0 0x7efc1819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 -- 172.21.15.99:0/3411599346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1819a0d0 con 0x7efc18103820 2024-05-14T17:00:57.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc16ffd700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 0x7efc1819f7b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc0800b530 tx=0x7efc0800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:57.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08015020 con 0x7efc18103820 2024-05-14T17:00:57.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1819a3c0 con 0x7efc18103820 2024-05-14T17:00:57.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc08011ce0 con 0x7efc18103820 2024-05-14T17:00:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08010dd0 con 0x7efc18103820 2024-05-14T17:00:57.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.649+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc181a6680 con 0x7efc18103820 2024-05-14T17:00:58.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.650+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1810bb50 con 0x7efc18103820 2024-05-14T17:00:58.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.651+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efc08018020 con 0x7efc18103820 2024-05-14T17:00:58.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.652+0000 7efc1c881700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7efc000b2990 0x7efc000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.652+0000 7efc167fc700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7efc000b2990 0x7efc000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.652+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efc080e1d70 con 0x7efc18103820 2024-05-14T17:00:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.653+0000 7efc167fc700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7efc000b2990 0x7efc000b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009990 tx=0x7efc0c0096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:58.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.655+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc0801f020 con 0x7efc18103820 2024-05-14T17:00:58.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.855+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc18063980 con 0x7efc000b2990 2024-05-14T17:00:58.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.862+0000 7efc1c881700 1 -- 172.21.15.99:0/3411599346 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258688 (secure 0 0 0) 0x7efc18063980 con 0x7efc000b2990 2024-05-14T17:00:58.073 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:00:58.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7efc000b2990 msgr2=0x7efc000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:58.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7efc000b2990 0x7efc000b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009990 tx=0x7efc0c0096d0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 msgr2=0x7efc1819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:58.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 0x7efc1819f7b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc0800b530 tx=0x7efc0800b840 comp rx=0 tx=0).stop 2024-05-14T17:00:58.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 shutdown_connections 2024-05-14T17:00:58.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7efc000b2990 0x7efc000b4e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc18108910 0x7efc18199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc18103820 0x7efc1819f7b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 --2- 172.21.15.99:0/3411599346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc181041d0 0x7efc1819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 >> 172.21.15.99:0/3411599346 conn(0x7efc1807a6b0 msgr2=0x7efc18102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:58.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 shutdown_connections 2024-05-14T17:00:58.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:00:57.865+0000 7efc1d883700 1 -- 172.21.15.99:0/3411599346 wait complete. 2024-05-14T17:00:58.172 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:00:58.500 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:58 smithi099 ceph-mon[31851]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T17:00:58.527 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-05-14T17:00:56.468608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650693,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":266,"num_read_kb":226,"num_write":287,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365508,"kb_used_data":12196,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576316,"statfs":{"total":1151940427776,"available":1151566147584,"internally_reserved":0,"allocated":12488704,"data_stored":6936228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14006,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":50,"num_write":13,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003423"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067089+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:00:12.067089+0000","last_peered":"2024-05-14T17:00:12.067089+0000","last_clean":"2024-05-14T17:00:12.067089+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:00:12.067089+0000","last_undegraded":"2024-05-14T17:00:12.067089+0000","last_fullsized":"2024-05-14T17:00:12.067089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066684+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:00:12.066684+0000","last_peered":"2024-05-14T17:00:12.066684+0000","last_clean":"2024-05-14T17:00:12.066684+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:00:12.066684+0000","last_undegraded":"2024-05-14T17:00:12.066684+0000","last_fullsized":"2024-05-14T17:00:12.066684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066609+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:00:12.066609+0000","last_peered":"2024-05-14T17:00:12.066609+0000","last_clean":"2024-05-14T17:00:12.066609+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:00:12.066609+0000","last_undegraded":"2024-05-14T17:00:12.066609+0000","last_fullsized":"2024-05-14T17:00:12.066609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063154+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:00:12.063154+0000","last_peered":"2024-05-14T17:00:12.063154+0000","last_clean":"2024-05-14T17:00:12.063154+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:00:12.063154+0000","last_undegraded":"2024-05-14T17:00:12.063154+0000","last_fullsized":"2024-05-14T17:00:12.063154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063448+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:00:12.063448+0000","last_peered":"2024-05-14T17:00:12.063448+0000","last_clean":"2024-05-14T17:00:12.063448+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:00:12.063448+0000","last_undegraded":"2024-05-14T17:00:12.063448+0000","last_fullsized":"2024-05-14T17:00:12.063448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062788+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:00:12.062788+0000","last_peered":"2024-05-14T17:00:12.062788+0000","last_clean":"2024-05-14T17:00:12.062788+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:00:12.062788+0000","last_undegraded":"2024-05-14T17:00:12.062788+0000","last_fullsized":"2024-05-14T17:00:12.062788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066581+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:00:12.066581+0000","last_peered":"2024-05-14T17:00:12.066581+0000","last_clean":"2024-05-14T17:00:12.066581+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:00:12.066581+0000","last_undegraded":"2024-05-14T17:00:12.066581+0000","last_fullsized":"2024-05-14T17:00:12.066581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055561+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:00:12.055561+0000","last_peered":"2024-05-14T17:00:12.055561+0000","last_clean":"2024-05-14T17:00:12.055561+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:00:12.055561+0000","last_undegraded":"2024-05-14T17:00:12.055561+0000","last_fullsized":"2024-05-14T17:00:12.055561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058946+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:00:12.058946+0000","last_peered":"2024-05-14T17:00:12.058946+0000","last_clean":"2024-05-14T17:00:12.058946+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:00:12.058946+0000","last_undegraded":"2024-05-14T17:00:12.058946+0000","last_fullsized":"2024-05-14T17:00:12.058946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063879+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:00:12.063879+0000","last_peered":"2024-05-14T17:00:12.063879+0000","last_clean":"2024-05-14T17:00:12.063879+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:00:12.063879+0000","last_undegraded":"2024-05-14T17:00:12.063879+0000","last_fullsized":"2024-05-14T17:00:12.063879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067726+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:00:12.067726+0000","last_peered":"2024-05-14T17:00:12.067726+0000","last_clean":"2024-05-14T17:00:12.067726+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:00:12.067726+0000","last_undegraded":"2024-05-14T17:00:12.067726+0000","last_fullsized":"2024-05-14T17:00:12.067726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062996+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:00:12.062996+0000","last_peered":"2024-05-14T17:00:12.062996+0000","last_clean":"2024-05-14T17:00:12.062996+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:00:12.062996+0000","last_undegraded":"2024-05-14T17:00:12.062996+0000","last_fullsized":"2024-05-14T17:00:12.062996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063365+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:00:12.063365+0000","last_peered":"2024-05-14T17:00:12.063365+0000","last_clean":"2024-05-14T17:00:12.063365+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:00:12.063365+0000","last_undegraded":"2024-05-14T17:00:12.063365+0000","last_fullsized":"2024-05-14T17:00:12.063365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.065791+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:00:12.065791+0000","last_peered":"2024-05-14T17:00:12.065791+0000","last_clean":"2024-05-14T17:00:12.065791+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:00:12.065791+0000","last_undegraded":"2024-05-14T17:00:12.065791+0000","last_fullsized":"2024-05-14T17:00:12.065791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066460+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:00:12.066460+0000","last_peered":"2024-05-14T17:00:12.066460+0000","last_clean":"2024-05-14T17:00:12.066460+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:00:12.066460+0000","last_undegraded":"2024-05-14T17:00:12.066460+0000","last_fullsized":"2024-05-14T17:00:12.066460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058251+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:00:12.058251+0000","last_peered":"2024-05-14T17:00:12.058251+0000","last_clean":"2024-05-14T17:00:12.058251+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:00:12.058251+0000","last_undegraded":"2024-05-14T17:00:12.058251+0000","last_fullsized":"2024-05-14T17:00:12.058251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067247+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:00:12.067247+0000","last_peered":"2024-05-14T17:00:12.067247+0000","last_clean":"2024-05-14T17:00:12.067247+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:00:12.067247+0000","last_undegraded":"2024-05-14T17:00:12.067247+0000","last_fullsized":"2024-05-14T17:00:12.067247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062756+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:00:12.062756+0000","last_peered":"2024-05-14T17:00:12.062756+0000","last_clean":"2024-05-14T17:00:12.062756+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:00:12.062756+0000","last_undegraded":"2024-05-14T17:00:12.062756+0000","last_fullsized":"2024-05-14T17:00:12.062756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.064067+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:00:12.064067+0000","last_peered":"2024-05-14T17:00:12.064067+0000","last_clean":"2024-05-14T17:00:12.064067+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:00:12.064067+0000","last_undegraded":"2024-05-14T17:00:12.064067+0000","last_fullsized":"2024-05-14T17:00:12.064067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125952+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:00:26.125952+0000","last_peered":"2024-05-14T17:00:26.125952+0000","last_clean":"2024-05-14T17:00:26.125952+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:00:26.125952+0000","last_undegraded":"2024-05-14T17:00:26.125952+0000","last_fullsized":"2024-05-14T17:00:26.125952+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055761+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:00:12.055761+0000","last_peered":"2024-05-14T17:00:12.055761+0000","last_clean":"2024-05-14T17:00:12.055761+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:00:12.055761+0000","last_undegraded":"2024-05-14T17:00:12.055761+0000","last_fullsized":"2024-05-14T17:00:12.055761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.065500+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:00:12.065500+0000","last_peered":"2024-05-14T17:00:12.065500+0000","last_clean":"2024-05-14T17:00:12.065500+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:00:12.065500+0000","last_undegraded":"2024-05-14T17:00:12.065500+0000","last_fullsized":"2024-05-14T17:00:12.065500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066866+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:00:12.066866+0000","last_peered":"2024-05-14T17:00:12.066866+0000","last_clean":"2024-05-14T17:00:12.066866+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:00:12.066866+0000","last_undegraded":"2024-05-14T17:00:12.066866+0000","last_fullsized":"2024-05-14T17:00:12.066866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.124887+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:00:26.124887+0000","last_peered":"2024-05-14T17:00:26.124887+0000","last_clean":"2024-05-14T17:00:26.124887+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:00:26.124887+0000","last_undegraded":"2024-05-14T17:00:26.124887+0000","last_fullsized":"2024-05-14T17:00:26.124887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058480+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:00:12.058480+0000","last_peered":"2024-05-14T17:00:12.058480+0000","last_clean":"2024-05-14T17:00:12.058480+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:00:12.058480+0000","last_undegraded":"2024-05-14T17:00:12.058480+0000","last_fullsized":"2024-05-14T17:00:12.058480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.070478+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:00:12.070478+0000","last_peered":"2024-05-14T17:00:12.070478+0000","last_clean":"2024-05-14T17:00:12.070478+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:00:12.070478+0000","last_undegraded":"2024-05-14T17:00:12.070478+0000","last_fullsized":"2024-05-14T17:00:12.070478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056810+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:00:12.056810+0000","last_peered":"2024-05-14T17:00:12.056810+0000","last_clean":"2024-05-14T17:00:12.056810+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:00:12.056810+0000","last_undegraded":"2024-05-14T17:00:12.056810+0000","last_fullsized":"2024-05-14T17:00:12.056810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":57,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:52.176688+0000","last_change":"2024-05-14T17:00:07.040846+0000","last_active":"2024-05-14T17:00:52.176688+0000","last_peered":"2024-05-14T17:00:52.176688+0000","last_clean":"2024-05-14T17:00:52.176688+0000","last_became_active":"2024-05-14T17:00:07.040449+0000","last_became_peered":"2024-05-14T17:00:07.040449+0000","last_unstale":"2024-05-14T17:00:52.176688+0000","last_undegraded":"2024-05-14T17:00:52.176688+0000","last_fullsized":"2024-05-14T17:00:52.176688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:01:52.192689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063647+0000","last_change":"2024-05-14T17:00:10.054669+0000","last_active":"2024-05-14T17:00:12.063647+0000","last_peered":"2024-05-14T17:00:12.063647+0000","last_clean":"2024-05-14T17:00:12.063647+0000","last_became_active":"2024-05-14T17:00:10.054298+0000","last_became_peered":"2024-05-14T17:00:10.054298+0000","last_unstale":"2024-05-14T17:00:12.063647+0000","last_undegraded":"2024-05-14T17:00:12.063647+0000","last_fullsized":"2024-05-14T17:00:12.063647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:35:54.172686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.124142+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:00:26.124142+0000","last_peered":"2024-05-14T17:00:26.124142+0000","last_clean":"2024-05-14T17:00:26.124142+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:00:26.124142+0000","last_undegraded":"2024-05-14T17:00:26.124142+0000","last_fullsized":"2024-05-14T17:00:26.124142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056898+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:00:12.056898+0000","last_peered":"2024-05-14T17:00:12.056898+0000","last_clean":"2024-05-14T17:00:12.056898+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:00:12.056898+0000","last_undegraded":"2024-05-14T17:00:12.056898+0000","last_fullsized":"2024-05-14T17:00:12.056898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.124887+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:00:26.124887+0000","last_peered":"2024-05-14T17:00:26.124887+0000","last_clean":"2024-05-14T17:00:26.124887+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:00:26.124887+0000","last_undegraded":"2024-05-14T17:00:26.124887+0000","last_fullsized":"2024-05-14T17:00:26.124887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055644+0000","last_change":"2024-05-14T17:00:10.058307+0000","last_active":"2024-05-14T17:00:12.055644+0000","last_peered":"2024-05-14T17:00:12.055644+0000","last_clean":"2024-05-14T17:00:12.055644+0000","last_became_active":"2024-05-14T17:00:10.058016+0000","last_became_peered":"2024-05-14T17:00:10.058016+0000","last_unstale":"2024-05-14T17:00:12.055644+0000","last_undegraded":"2024-05-14T17:00:12.055644+0000","last_fullsized":"2024-05-14T17:00:12.055644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:51.000609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056328+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:00:12.056328+0000","last_peered":"2024-05-14T17:00:12.056328+0000","last_clean":"2024-05-14T17:00:12.056328+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:00:12.056328+0000","last_undegraded":"2024-05-14T17:00:12.056328+0000","last_fullsized":"2024-05-14T17:00:12.056328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.061983+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:00:12.061983+0000","last_peered":"2024-05-14T17:00:12.061983+0000","last_clean":"2024-05-14T17:00:12.061983+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:00:12.061983+0000","last_undegraded":"2024-05-14T17:00:12.061983+0000","last_fullsized":"2024-05-14T17:00:12.061983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":51,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:52.176216+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:00:52.176216+0000","last_peered":"2024-05-14T17:00:52.176216+0000","last_clean":"2024-05-14T17:00:52.176216+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:00:52.176216+0000","last_undegraded":"2024-05-14T17:00:52.176216+0000","last_fullsized":"2024-05-14T17:00:52.176216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058894+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:00:12.058894+0000","last_peered":"2024-05-14T17:00:12.058894+0000","last_clean":"2024-05-14T17:00:12.058894+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:00:12.058894+0000","last_undegraded":"2024-05-14T17:00:12.058894+0000","last_fullsized":"2024-05-14T17:00:12.058894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119686+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:00:25.119686+0000","last_peered":"2024-05-14T17:00:25.119686+0000","last_clean":"2024-05-14T17:00:25.119686+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:00:25.119686+0000","last_undegraded":"2024-05-14T17:00:25.119686+0000","last_fullsized":"2024-05-14T17:00:25.119686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066819+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:00:12.066819+0000","last_peered":"2024-05-14T17:00:12.066819+0000","last_clean":"2024-05-14T17:00:12.066819+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:00:12.066819+0000","last_undegraded":"2024-05-14T17:00:12.066819+0000","last_fullsized":"2024-05-14T17:00:12.066819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.118724+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:00:25.118724+0000","last_peered":"2024-05-14T17:00:25.118724+0000","last_clean":"2024-05-14T17:00:25.118724+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:00:25.118724+0000","last_undegraded":"2024-05-14T17:00:25.118724+0000","last_fullsized":"2024-05-14T17:00:25.118724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058608+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:00:12.058608+0000","last_peered":"2024-05-14T17:00:12.058608+0000","last_clean":"2024-05-14T17:00:12.058608+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:00:12.058608+0000","last_undegraded":"2024-05-14T17:00:12.058608+0000","last_fullsized":"2024-05-14T17:00:12.058608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'27","reported_seq":109,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:53.177512+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:00:53.177512+0000","last_peered":"2024-05-14T17:00:53.177512+0000","last_clean":"2024-05-14T17:00:53.177512+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:00:53.177512+0000","last_undegraded":"2024-05-14T17:00:53.177512+0000","last_fullsized":"2024-05-14T17:00:53.177512+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56090,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":60,"num_read_kb":84,"num_write":28,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063582+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:00:12.063582+0000","last_peered":"2024-05-14T17:00:12.063582+0000","last_clean":"2024-05-14T17:00:12.063582+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:00:12.063582+0000","last_undegraded":"2024-05-14T17:00:12.063582+0000","last_fullsized":"2024-05-14T17:00:12.063582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125102+0000","last_change":"2024-05-14T17:00:07.038589+0000","last_active":"2024-05-14T17:00:26.125102+0000","last_peered":"2024-05-14T17:00:26.125102+0000","last_clean":"2024-05-14T17:00:26.125102+0000","last_became_active":"2024-05-14T17:00:07.037870+0000","last_became_peered":"2024-05-14T17:00:07.037870+0000","last_unstale":"2024-05-14T17:00:26.125102+0000","last_undegraded":"2024-05-14T17:00:26.125102+0000","last_fullsized":"2024-05-14T17:00:26.125102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:49:21.471467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062915+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:00:12.062915+0000","last_peered":"2024-05-14T17:00:12.062915+0000","last_clean":"2024-05-14T17:00:12.062915+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:00:12.062915+0000","last_undegraded":"2024-05-14T17:00:12.062915+0000","last_fullsized":"2024-05-14T17:00:12.062915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119040+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:00:25.119040+0000","last_peered":"2024-05-14T17:00:25.119040+0000","last_clean":"2024-05-14T17:00:25.119040+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:00:25.119040+0000","last_undegraded":"2024-05-14T17:00:25.119040+0000","last_fullsized":"2024-05-14T17:00:25.119040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067254+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:00:12.067254+0000","last_peered":"2024-05-14T17:00:12.067254+0000","last_clean":"2024-05-14T17:00:12.067254+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:00:12.067254+0000","last_undegraded":"2024-05-14T17:00:12.067254+0000","last_fullsized":"2024-05-14T17:00:12.067254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:52.170873+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:00:52.170873+0000","last_peered":"2024-05-14T17:00:52.170873+0000","last_clean":"2024-05-14T17:00:52.170873+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:00:52.170873+0000","last_undegraded":"2024-05-14T17:00:52.170873+0000","last_fullsized":"2024-05-14T17:00:52.170873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066827+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:00:12.066827+0000","last_peered":"2024-05-14T17:00:12.066827+0000","last_clean":"2024-05-14T17:00:12.066827+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:00:12.066827+0000","last_undegraded":"2024-05-14T17:00:12.066827+0000","last_fullsized":"2024-05-14T17:00:12.066827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.127428+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:00:26.127428+0000","last_peered":"2024-05-14T17:00:26.127428+0000","last_clean":"2024-05-14T17:00:26.127428+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:00:26.127428+0000","last_undegraded":"2024-05-14T17:00:26.127428+0000","last_fullsized":"2024-05-14T17:00:26.127428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056397+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:00:12.056397+0000","last_peered":"2024-05-14T17:00:12.056397+0000","last_clean":"2024-05-14T17:00:12.056397+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:00:12.056397+0000","last_undegraded":"2024-05-14T17:00:12.056397+0000","last_fullsized":"2024-05-14T17:00:12.056397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:28.121726+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:00:28.121726+0000","last_peered":"2024-05-14T17:00:28.121726+0000","last_clean":"2024-05-14T17:00:28.121726+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:00:28.121726+0000","last_undegraded":"2024-05-14T17:00:28.121726+0000","last_fullsized":"2024-05-14T17:00:28.121726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066947+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:00:12.066947+0000","last_peered":"2024-05-14T17:00:12.066947+0000","last_clean":"2024-05-14T17:00:12.066947+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:00:12.066947+0000","last_undegraded":"2024-05-14T17:00:12.066947+0000","last_fullsized":"2024-05-14T17:00:12.066947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.061816+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:00:12.061816+0000","last_peered":"2024-05-14T17:00:12.061816+0000","last_clean":"2024-05-14T17:00:12.061816+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:00:12.061816+0000","last_undegraded":"2024-05-14T17:00:12.061816+0000","last_fullsized":"2024-05-14T17:00:12.061816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:13.923463+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:00:13.923463+0000","last_peered":"2024-05-14T17:00:13.923463+0000","last_clean":"2024-05-14T17:00:13.923463+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:00:13.923463+0000","last_undegraded":"2024-05-14T17:00:13.923463+0000","last_fullsized":"2024-05-14T17:00:13.923463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.068021+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:00:12.068021+0000","last_peered":"2024-05-14T17:00:12.068021+0000","last_clean":"2024-05-14T17:00:12.068021+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:00:12.068021+0000","last_undegraded":"2024-05-14T17:00:12.068021+0000","last_fullsized":"2024-05-14T17:00:12.068021+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062915+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:00:12.062915+0000","last_peered":"2024-05-14T17:00:12.062915+0000","last_clean":"2024-05-14T17:00:12.062915+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:00:12.062915+0000","last_undegraded":"2024-05-14T17:00:12.062915+0000","last_fullsized":"2024-05-14T17:00:12.062915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056628+0000","last_change":"2024-05-14T17:00:10.060967+0000","last_active":"2024-05-14T17:00:12.056628+0000","last_peered":"2024-05-14T17:00:12.056628+0000","last_clean":"2024-05-14T17:00:12.056628+0000","last_became_active":"2024-05-14T17:00:10.060392+0000","last_became_peered":"2024-05-14T17:00:10.060392+0000","last_unstale":"2024-05-14T17:00:12.056628+0000","last_undegraded":"2024-05-14T17:00:12.056628+0000","last_fullsized":"2024-05-14T17:00:12.056628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:25:13.345462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":54,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:54.173542+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:00:54.173542+0000","last_peered":"2024-05-14T17:00:54.173542+0000","last_clean":"2024-05-14T17:00:54.173542+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:00:54.173542+0000","last_undegraded":"2024-05-14T17:00:54.173542+0000","last_fullsized":"2024-05-14T17:00:54.173542+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056070+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:00:12.056070+0000","last_peered":"2024-05-14T17:00:12.056070+0000","last_clean":"2024-05-14T17:00:12.056070+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:00:12.056070+0000","last_undegraded":"2024-05-14T17:00:12.056070+0000","last_fullsized":"2024-05-14T17:00:12.056070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:28.132394+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:00:28.132394+0000","last_peered":"2024-05-14T17:00:28.132394+0000","last_clean":"2024-05-14T17:00:28.132394+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:00:28.132394+0000","last_undegraded":"2024-05-14T17:00:28.132394+0000","last_fullsized":"2024-05-14T17:00:28.132394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067036+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:00:12.067036+0000","last_peered":"2024-05-14T17:00:12.067036+0000","last_clean":"2024-05-14T17:00:12.067036+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:00:12.067036+0000","last_undegraded":"2024-05-14T17:00:12.067036+0000","last_fullsized":"2024-05-14T17:00:12.067036+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.062530+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:00:12.062530+0000","last_peered":"2024-05-14T17:00:12.062530+0000","last_clean":"2024-05-14T17:00:12.062530+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:00:12.062530+0000","last_undegraded":"2024-05-14T17:00:12.062530+0000","last_fullsized":"2024-05-14T17:00:12.062530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125897+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:00:26.125897+0000","last_peered":"2024-05-14T17:00:26.125897+0000","last_clean":"2024-05-14T17:00:26.125897+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:00:26.125897+0000","last_undegraded":"2024-05-14T17:00:26.125897+0000","last_fullsized":"2024-05-14T17:00:26.125897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.117959+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:00:25.117959+0000","last_peered":"2024-05-14T17:00:25.117959+0000","last_clean":"2024-05-14T17:00:25.117959+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:00:25.117959+0000","last_undegraded":"2024-05-14T17:00:25.117959+0000","last_fullsized":"2024-05-14T17:00:25.117959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063284+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:00:12.063284+0000","last_peered":"2024-05-14T17:00:12.063284+0000","last_clean":"2024-05-14T17:00:12.063284+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:00:12.063284+0000","last_undegraded":"2024-05-14T17:00:12.063284+0000","last_fullsized":"2024-05-14T17:00:12.063284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:48.065763+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:00:48.065763+0000","last_peered":"2024-05-14T17:00:48.065763+0000","last_clean":"2024-05-14T17:00:48.065763+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:00:48.065763+0000","last_undegraded":"2024-05-14T17:00:48.065763+0000","last_fullsized":"2024-05-14T17:00:48.065763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056049+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:00:12.056049+0000","last_peered":"2024-05-14T17:00:12.056049+0000","last_clean":"2024-05-14T17:00:12.056049+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:00:12.056049+0000","last_undegraded":"2024-05-14T17:00:12.056049+0000","last_fullsized":"2024-05-14T17:00:12.056049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.118018+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:00:25.118018+0000","last_peered":"2024-05-14T17:00:25.118018+0000","last_clean":"2024-05-14T17:00:25.118018+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:00:25.118018+0000","last_undegraded":"2024-05-14T17:00:25.118018+0000","last_fullsized":"2024-05-14T17:00:25.118018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066868+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:00:12.066868+0000","last_peered":"2024-05-14T17:00:12.066868+0000","last_clean":"2024-05-14T17:00:12.066868+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:00:12.066868+0000","last_undegraded":"2024-05-14T17:00:12.066868+0000","last_fullsized":"2024-05-14T17:00:12.066868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119381+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:00:25.119381+0000","last_peered":"2024-05-14T17:00:25.119381+0000","last_clean":"2024-05-14T17:00:25.119381+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:00:25.119381+0000","last_undegraded":"2024-05-14T17:00:25.119381+0000","last_fullsized":"2024-05-14T17:00:25.119381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.064843+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:00:12.064843+0000","last_peered":"2024-05-14T17:00:12.064843+0000","last_clean":"2024-05-14T17:00:12.064843+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:00:12.064843+0000","last_undegraded":"2024-05-14T17:00:12.064843+0000","last_fullsized":"2024-05-14T17:00:12.064843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.117355+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:00:25.117355+0000","last_peered":"2024-05-14T17:00:25.117355+0000","last_clean":"2024-05-14T17:00:25.117355+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:00:25.117355+0000","last_undegraded":"2024-05-14T17:00:25.117355+0000","last_fullsized":"2024-05-14T17:00:25.117355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067409+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:00:12.067409+0000","last_peered":"2024-05-14T17:00:12.067409+0000","last_clean":"2024-05-14T17:00:12.067409+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:00:12.067409+0000","last_undegraded":"2024-05-14T17:00:12.067409+0000","last_fullsized":"2024-05-14T17:00:12.067409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063948+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:00:12.063948+0000","last_peered":"2024-05-14T17:00:12.063948+0000","last_clean":"2024-05-14T17:00:12.063948+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:00:12.063948+0000","last_undegraded":"2024-05-14T17:00:12.063948+0000","last_fullsized":"2024-05-14T17:00:12.063948+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055792+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:00:12.055792+0000","last_peered":"2024-05-14T17:00:12.055792+0000","last_clean":"2024-05-14T17:00:12.055792+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:00:12.055792+0000","last_undegraded":"2024-05-14T17:00:12.055792+0000","last_fullsized":"2024-05-14T17:00:12.055792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:25.119414+0000","last_change":"2024-05-14T17:00:07.043988+0000","last_active":"2024-05-14T17:00:25.119414+0000","last_peered":"2024-05-14T17:00:25.119414+0000","last_clean":"2024-05-14T17:00:25.119414+0000","last_became_active":"2024-05-14T17:00:07.043620+0000","last_became_peered":"2024-05-14T17:00:07.043620+0000","last_unstale":"2024-05-14T17:00:25.119414+0000","last_undegraded":"2024-05-14T17:00:25.119414+0000","last_fullsized":"2024-05-14T17:00:25.119414+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:03.650546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067758+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:00:12.067758+0000","last_peered":"2024-05-14T17:00:12.067758+0000","last_clean":"2024-05-14T17:00:12.067758+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:00:12.067758+0000","last_undegraded":"2024-05-14T17:00:12.067758+0000","last_fullsized":"2024-05-14T17:00:12.067758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:26.125373+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:00:26.125373+0000","last_peered":"2024-05-14T17:00:26.125373+0000","last_clean":"2024-05-14T17:00:26.125373+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:00:26.125373+0000","last_undegraded":"2024-05-14T17:00:26.125373+0000","last_fullsized":"2024-05-14T17:00:26.125373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067995+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:00:12.067995+0000","last_peered":"2024-05-14T17:00:12.067995+0000","last_clean":"2024-05-14T17:00:12.067995+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:00:12.067995+0000","last_undegraded":"2024-05-14T17:00:12.067995+0000","last_fullsized":"2024-05-14T17:00:12.067995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.069089+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:00:12.069089+0000","last_peered":"2024-05-14T17:00:12.069089+0000","last_clean":"2024-05-14T17:00:12.069089+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:00:12.069089+0000","last_undegraded":"2024-05-14T17:00:12.069089+0000","last_fullsized":"2024-05-14T17:00:12.069089+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067634+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:00:12.067634+0000","last_peered":"2024-05-14T17:00:12.067634+0000","last_clean":"2024-05-14T17:00:12.067634+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:00:12.067634+0000","last_undegraded":"2024-05-14T17:00:12.067634+0000","last_fullsized":"2024-05-14T17:00:12.067634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.071718+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:00:12.071718+0000","last_peered":"2024-05-14T17:00:12.071718+0000","last_clean":"2024-05-14T17:00:12.071718+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:00:12.071718+0000","last_undegraded":"2024-05-14T17:00:12.071718+0000","last_fullsized":"2024-05-14T17:00:12.071718+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067340+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:00:12.067340+0000","last_peered":"2024-05-14T17:00:12.067340+0000","last_clean":"2024-05-14T17:00:12.067340+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:00:12.067340+0000","last_undegraded":"2024-05-14T17:00:12.067340+0000","last_fullsized":"2024-05-14T17:00:12.067340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.067643+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:00:12.067643+0000","last_peered":"2024-05-14T17:00:12.067643+0000","last_clean":"2024-05-14T17:00:12.067643+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:00:12.067643+0000","last_undegraded":"2024-05-14T17:00:12.067643+0000","last_fullsized":"2024-05-14T17:00:12.067643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063307+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:00:12.063307+0000","last_peered":"2024-05-14T17:00:12.063307+0000","last_clean":"2024-05-14T17:00:12.063307+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:00:12.063307+0000","last_undegraded":"2024-05-14T17:00:12.063307+0000","last_fullsized":"2024-05-14T17:00:12.063307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063682+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:00:12.063682+0000","last_peered":"2024-05-14T17:00:12.063682+0000","last_clean":"2024-05-14T17:00:12.063682+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:00:12.063682+0000","last_undegraded":"2024-05-14T17:00:12.063682+0000","last_fullsized":"2024-05-14T17:00:12.063682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066199+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:00:12.066199+0000","last_peered":"2024-05-14T17:00:12.066199+0000","last_clean":"2024-05-14T17:00:12.066199+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:00:12.066199+0000","last_undegraded":"2024-05-14T17:00:12.066199+0000","last_fullsized":"2024-05-14T17:00:12.066199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063441+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:00:12.063441+0000","last_peered":"2024-05-14T17:00:12.063441+0000","last_clean":"2024-05-14T17:00:12.063441+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:00:12.063441+0000","last_undegraded":"2024-05-14T17:00:12.063441+0000","last_fullsized":"2024-05-14T17:00:12.063441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066919+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:00:12.066919+0000","last_peered":"2024-05-14T17:00:12.066919+0000","last_clean":"2024-05-14T17:00:12.066919+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:00:12.066919+0000","last_undegraded":"2024-05-14T17:00:12.066919+0000","last_fullsized":"2024-05-14T17:00:12.066919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.064600+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:00:12.064600+0000","last_peered":"2024-05-14T17:00:12.064600+0000","last_clean":"2024-05-14T17:00:12.064600+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:00:12.064600+0000","last_undegraded":"2024-05-14T17:00:12.064600+0000","last_fullsized":"2024-05-14T17:00:12.064600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055821+0000","last_change":"2024-05-14T17:00:10.068405+0000","last_active":"2024-05-14T17:00:12.055821+0000","last_peered":"2024-05-14T17:00:12.055821+0000","last_clean":"2024-05-14T17:00:12.055821+0000","last_became_active":"2024-05-14T17:00:10.068075+0000","last_became_peered":"2024-05-14T17:00:10.068075+0000","last_unstale":"2024-05-14T17:00:12.055821+0000","last_undegraded":"2024-05-14T17:00:12.055821+0000","last_fullsized":"2024-05-14T17:00:12.055821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:49.394248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.056246+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:00:12.056246+0000","last_peered":"2024-05-14T17:00:12.056246+0000","last_clean":"2024-05-14T17:00:12.056246+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:00:12.056246+0000","last_undegraded":"2024-05-14T17:00:12.056246+0000","last_fullsized":"2024-05-14T17:00:12.056246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.055597+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:00:12.055597+0000","last_peered":"2024-05-14T17:00:12.055597+0000","last_clean":"2024-05-14T17:00:12.055597+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:00:12.055597+0000","last_undegraded":"2024-05-14T17:00:12.055597+0000","last_fullsized":"2024-05-14T17:00:12.055597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.058488+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:00:12.058488+0000","last_peered":"2024-05-14T17:00:12.058488+0000","last_clean":"2024-05-14T17:00:12.058488+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:00:12.058488+0000","last_undegraded":"2024-05-14T17:00:12.058488+0000","last_fullsized":"2024-05-14T17:00:12.058488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.063509+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:00:12.063509+0000","last_peered":"2024-05-14T17:00:12.063509+0000","last_clean":"2024-05-14T17:00:12.063509+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:00:12.063509+0000","last_undegraded":"2024-05-14T17:00:12.063509+0000","last_fullsized":"2024-05-14T17:00:12.063509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T17:00:12.066997+0000","last_change":"2024-05-14T17:00:10.059170+0000","last_active":"2024-05-14T17:00:12.066997+0000","last_peered":"2024-05-14T17:00:12.066997+0000","last_clean":"2024-05-14T17:00:12.066997+0000","last_became_active":"2024-05-14T17:00:10.057455+0000","last_became_peered":"2024-05-14T17:00:10.057455+0000","last_unstale":"2024-05-14T17:00:12.066997+0000","last_undegraded":"2024-05-14T17:00:12.066997+0000","last_fullsized":"2024-05-14T17:00:12.066997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:45.202488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60091,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":185,"num_read_kb":157,"num_write":162,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":184530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645337,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":415573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841565,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1063144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005090,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":415729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":1007659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":415198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":471081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":884736,"data_stored":416469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":477079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":415509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":415635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1482752,"data_stored":1007343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:00:58.571 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T17:00:58.579 INFO:tasks.cephadm:Setting up client nodes... 2024-05-14T17:00:58.600 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-14T17:00:58.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:58 smithi193 ceph-mon[35962]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T17:00:58.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:58 smithi150 ceph-mon[36204]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T17:00:58.871 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:00:59 smithi099 ceph-mon[31851]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:00:59.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:00:59 smithi193 ceph-mon[35962]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:00:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:00:59 smithi150 ceph-mon[36204]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:01:00.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 -- 172.21.15.99:0/395899221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 msgr2=0x7effa010ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:00.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 --2- 172.21.15.99:0/395899221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa010ba20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b0d0 tx=0x7eff9800b3e0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 -- 172.21.15.99:0/395899221 shutdown_connections 2024-05-14T17:01:00.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 --2- 172.21.15.99:0/395899221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa010ba20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 --2- 172.21.15.99:0/395899221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 0x7effa01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 --2- 172.21.15.99:0/395899221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa01047d0 0x7effa0104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 -- 172.21.15.99:0/395899221 >> 172.21.15.99:0/395899221 conn(0x7effa00feb90 msgr2=0x7effa0100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:00.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.469+0000 7effa7bb8700 1 -- 172.21.15.99:0/395899221 shutdown_connections 2024-05-14T17:01:00.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 -- 172.21.15.99:0/395899221 wait complete. 2024-05-14T17:01:00.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:00 smithi099 ceph-mon[31851]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-14T17:01:00.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:00 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:00.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 Processor -- start 2024-05-14T17:01:00.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 -- start start 2024-05-14T17:01:00.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 0x7effa0199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa01047d0 0x7effa0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effa010f070 con 0x7effa0105170 2024-05-14T17:01:00.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7effa010eef0 con 0x7effa01047d0 2024-05-14T17:01:00.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.470+0000 7effa7bb8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7effa010f1f0 con 0x7effa01035d0 2024-05-14T17:01:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 0x7effa0199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa6155700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa6155700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa01a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44720/0 (socket says 172.21.15.99:44720) 2024-05-14T17:01:00.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5153700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa01047d0 0x7effa0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa6155700 1 -- 172.21.15.99:0/4162097077 learned_addr learned my addr 172.21.15.99:0/4162097077 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:00.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 -- 172.21.15.99:0/4162097077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa01047d0 msgr2=0x7effa0199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:00.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa01047d0 0x7effa0199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 -- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 msgr2=0x7effa01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:00.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:00.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 -- 172.21.15.99:0/4162097077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01a11f0 con 0x7effa01035d0 2024-05-14T17:01:00.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa6155700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:00.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa5954700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 0x7effa0199590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00e9f0 tx=0x7eff9c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:00.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c004230 con 0x7effa01035d0 2024-05-14T17:01:00.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9c0043d0 con 0x7effa01035d0 2024-05-14T17:01:00.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01a14e0 con 0x7effa01035d0 2024-05-14T17:01:00.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.471+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c003d30 con 0x7effa01035d0 2024-05-14T17:01:00.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.472+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01a19d0 con 0x7effa01035d0 2024-05-14T17:01:00.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.473+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa0051a80 con 0x7effa01035d0 2024-05-14T17:01:00.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.474+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7eff9c00c7c0 con 0x7effa01035d0 2024-05-14T17:01:00.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.475+0000 7eff96ffd700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7eff8c0b2a30 0x7eff8c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:00.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.475+0000 7effa5153700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7eff8c0b2a30 0x7eff8c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:00.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.475+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7eff9c0e2590 con 0x7effa01035d0 2024-05-14T17:01:00.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.475+0000 7effa5153700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7eff8c0b2a30 0x7eff8c0b4ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff90007be0 tx=0x7eff900073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:00.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.479+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff9c0aa200 con 0x7effa01035d0 2024-05-14T17:01:00.955 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:00 smithi150 ceph-mon[36204]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-14T17:01:00.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:00 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:01.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:00 smithi193 ceph-mon[35962]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-14T17:01:01.014 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:00 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:01.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.798+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7effa002e6e0 con 0x7effa01035d0 2024-05-14T17:01:01.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.803+0000 7eff96ffd700 1 -- 172.21.15.99:0/4162097077 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7effa002e6e0 con 0x7effa01035d0 2024-05-14T17:01:01.036 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-05-14T17:01:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDMmENmXie2LxAAlwI7XKqVnF/fRNC/6DhFTw== 2024-05-14T17:01:01.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.806+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7eff8c0b2a30 msgr2=0x7eff8c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:01.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.806+0000 7effa7bb8700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7eff8c0b2a30 0x7eff8c0b4ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7eff90007be0 tx=0x7eff900073d0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.806+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 msgr2=0x7effa0199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:01.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.806+0000 7effa7bb8700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 0x7effa0199590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00e9f0 tx=0x7eff9c00ed00 comp rx=0 tx=0).stop 2024-05-14T17:01:01.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 shutdown_connections 2024-05-14T17:01:01.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7eff8c0b2a30 0x7eff8c0b4ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa0105170 0x7effa01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effa01035d0 0x7effa0199590 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 --2- 172.21.15.99:0/4162097077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa01047d0 0x7effa0199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:01.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 >> 172.21.15.99:0/4162097077 conn(0x7effa00feb90 msgr2=0x7effa0107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:01.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 shutdown_connections 2024-05-14T17:01:01.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:00.807+0000 7effa7bb8700 1 -- 172.21.15.99:0/4162097077 wait complete. 2024-05-14T17:01:01.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:01 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4162097077' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T17:01:01.253 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:01 smithi193 ceph-mon[35962]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T17:01:01.259 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:01 smithi193 ceph-mon[35962]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T17:01:01.341 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:01 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4162097077' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T17:01:01.351 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:01 smithi150 ceph-mon[36204]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T17:01:01.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:01 smithi150 ceph-mon[36204]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T17:01:01.378 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:01 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4162097077' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T17:01:01.385 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:01 smithi099 ceph-mon[31851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T17:01:01.391 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:01 smithi099 ceph-mon[31851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T17:01:01.398 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T17:01:01.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-14T17:01:01.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-14T17:01:01.465 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-14T17:01:01.472 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-14T17:01:01.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-14T17:01:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:02 smithi099 ceph-mon[31851]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-14T17:01:02.523 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.556 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.570 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:02 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.577 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:02 smithi193 ceph-mon[35962]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-14T17:01:02.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.611 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.618 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:02 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:02 smithi150 ceph-mon[36204]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-14T17:01:02.711 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.730 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.737 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:02.744 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:02 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:03.439 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-14T17:01:03.447 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-14T17:01:03.493 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-14T17:01:03.501 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7, 5, 9]}]: dispatch 2024-05-14T17:01:03.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-05-14T17:01:03.516 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-14T17:01:03.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-05-14T17:01:03.528 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:03 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-14T17:01:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-14T17:01:03.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-14T17:01:03.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-14T17:01:03.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7, 5, 9]}]: dispatch 2024-05-14T17:01:03.595 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-05-14T17:01:03.602 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-14T17:01:03.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-05-14T17:01:03.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:03 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-14T17:01:03.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 -- 172.21.15.99:0/4114090975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd941028c0 msgr2=0x7fbd94110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 --2- 172.21.15.99:0/4114090975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd941028c0 0x7fbd94110330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00b0a0 tx=0x7fbd8c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 -- 172.21.15.99:0/4114090975 shutdown_connections 2024-05-14T17:01:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 --2- 172.21.15.99:0/4114090975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd941028c0 0x7fbd94110330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 --2- 172.21.15.99:0/4114090975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd94108620 0x7fbd94108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 --2- 172.21.15.99:0/4114090975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd94101f00 0x7fbd94102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:03.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 -- 172.21.15.99:0/4114090975 >> 172.21.15.99:0/4114090975 conn(0x7fbd94077780 msgr2=0x7fbd94077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:03.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 -- 172.21.15.99:0/4114090975 shutdown_connections 2024-05-14T17:01:03.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.668+0000 7fbd9997a700 1 -- 172.21.15.99:0/4114090975 wait complete. 2024-05-14T17:01:03.752 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-14T17:01:03.782 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-14T17:01:03.789 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-14T17:01:03.796 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7, 5, 9]}]: dispatch 2024-05-14T17:01:03.803 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-05-14T17:01:03.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-05-14T17:01:03.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-05-14T17:01:03.825 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:03 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-14T17:01:03.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 Processor -- start 2024-05-14T17:01:03.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 -- start start 2024-05-14T17:01:03.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 0x7fbd9419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:03.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd941028c0 0x7fbd9419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:03.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd94108620 0x7fbd94199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:03.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941137d0 con 0x7fbd94101f00 2024-05-14T17:01:03.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94113650 con 0x7fbd941028c0 2024-05-14T17:01:03.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd9997a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94113950 con 0x7fbd94108620 2024-05-14T17:01:03.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd92ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 0x7fbd9419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:03.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd92ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 0x7fbd9419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44736/0 (socket says 172.21.15.99:44736) 2024-05-14T17:01:03.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd92ffd700 1 -- 172.21.15.99:0/1464867542 learned_addr learned my addr 172.21.15.99:0/1464867542 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:03.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd937fe700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd94108620 0x7fbd94199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:03.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.669+0000 7fbd927fc700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd941028c0 0x7fbd9419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:03.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd92ffd700 1 -- 172.21.15.99:0/1464867542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd94108620 msgr2=0x7fbd94199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd92ffd700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd94108620 0x7fbd94199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd92ffd700 1 -- 172.21.15.99:0/1464867542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd941028c0 msgr2=0x7fbd9419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:04.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd92ffd700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd941028c0 0x7fbd9419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd92ffd700 1 -- 172.21.15.99:0/1464867542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9419a290 con 0x7fbd94101f00 2024-05-14T17:01:04.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd927fc700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd941028c0 0x7fbd9419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:04.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd92ffd700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 0x7fbd9419f920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b500 tx=0x7fbd8800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88015020 con 0x7fbd94101f00 2024-05-14T17:01:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd88011ce0 con 0x7fbd94101f00 2024-05-14T17:01:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9419a580 con 0x7fbd94101f00 2024-05-14T17:01:04.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88010d20 con 0x7fbd94101f00 2024-05-14T17:01:04.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.670+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941a67f0 con 0x7fbd94101f00 2024-05-14T17:01:04.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.671+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9406a230 con 0x7fbd94101f00 2024-05-14T17:01:04.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.672+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbd88018020 con 0x7fbd94101f00 2024-05-14T17:01:04.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.673+0000 7fbd98978700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbd7c0b2910 0x7fbd7c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:04.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.673+0000 7fbd927fc700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbd7c0b2910 0x7fbd7c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:04.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.673+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fbd880e1f70 con 0x7fbd94101f00 2024-05-14T17:01:04.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.673+0000 7fbd927fc700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbd7c0b2910 0x7fbd7c0b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd84005fd0 tx=0x7fbd84005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:04.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.676+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd8801f020 con 0x7fbd94101f00 2024-05-14T17:01:04.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.983+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbd9402e560 con 0x7fbd94101f00 2024-05-14T17:01:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.988+0000 7fbd98978700 1 -- 172.21.15.99:0/1464867542 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288645 (secure 0 0 0) 0x7fbd880a9c50 con 0x7fbd94101f00 2024-05-14T17:01:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:01:04.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbd7c0b2910 msgr2=0x7fbd7c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:04.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbd7c0b2910 0x7fbd7c0b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd84005fd0 tx=0x7fbd84005ee0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 msgr2=0x7fbd9419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:04.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 0x7fbd9419f920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b500 tx=0x7fbd8800b810 comp rx=0 tx=0).stop 2024-05-14T17:01:04.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 shutdown_connections 2024-05-14T17:01:04.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbd7c0b2910 0x7fbd7c0b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd94108620 0x7fbd94199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd94101f00 0x7fbd9419f920 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 --2- 172.21.15.99:0/1464867542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd941028c0 0x7fbd9419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:04.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 >> 172.21.15.99:0/1464867542 conn(0x7fbd94077780 msgr2=0x7fbd941009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:04.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 shutdown_connections 2024-05-14T17:01:04.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:03.992+0000 7fbd9997a700 1 -- 172.21.15.99:0/1464867542 wait complete. 2024-05-14T17:01:04.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-14T17:01:04.527 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-14T17:01:04.534 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-14T17:01:04.542 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-14T17:01:04.551 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7, 5, 9]}]': finished 2024-05-14T17:01:04.567 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-05-14T17:01:04.575 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-14T17:01:04.594 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-05-14T17:01:04.602 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-14T17:01:04.610 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T17:01:04.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:04.624 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:04.630 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:04 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1464867542' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T17:01:04.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-14T17:01:04.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-14T17:01:04.670 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-14T17:01:04.684 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-14T17:01:04.692 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7, 5, 9]}]': finished 2024-05-14T17:01:04.699 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-05-14T17:01:04.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-14T17:01:04.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-05-14T17:01:04.718 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-14T17:01:04.750 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T17:01:04.757 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:04.778 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:04.786 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:04 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1464867542' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T17:01:04.795 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2498052825},{"type":"v1","addr":"172.21.15.99:6801","nonce":2498052825}]},"active_addr":"172.21.15.99:6801/2498052825","active_change":"2024-05-14T16:53:02.369668+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.99:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":781830097}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1935223933}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2842786850}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":701736301}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2519097608}]}]} 2024-05-14T17:01:04.820 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-14T17:01:04.828 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-14T17:01:04.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:01:04.844 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-14T17:01:04.856 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-14T17:01:04.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-14T17:01:04.892 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-14T17:01:04.899 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7, 5, 9]}]': finished 2024-05-14T17:01:04.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-05-14T17:01:04.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-05-14T17:01:04.921 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-05-14T17:01:04.928 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-14T17:01:04.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T17:01:04.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:04.985 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:05.019 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:04 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1464867542' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T17:01:05.264 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:05.387 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:05 smithi099 ceph-mon[31851]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T17:01:05.395 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:05.401 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:05.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:05 smithi193 ceph-mon[35962]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T17:01:05.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:05.568 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:05.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:05.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:05 smithi150 ceph-mon[36204]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T17:01:05.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:05.719 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:05.726 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:01:06.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:06 smithi099 ceph-mon[31851]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-14T17:01:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:06 smithi193 ceph-mon[35962]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-14T17:01:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:06 smithi150 ceph-mon[36204]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-05-14T17:01:06.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 -- 172.21.15.99:0/946520595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103600 msgr2=0x7f1204103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:06.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 --2- 172.21.15.99:0/946520595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103600 0x7f1204103a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0099c0 tx=0x7f11ec009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 -- 172.21.15.99:0/946520595 shutdown_connections 2024-05-14T17:01:06.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 --2- 172.21.15.99:0/946520595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1204103fc0 0x7f12041103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 --2- 172.21.15.99:0/946520595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12041095e0 0x7f12041099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 --2- 172.21.15.99:0/946520595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103600 0x7f1204103a80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 -- 172.21.15.99:0/946520595 >> 172.21.15.99:0/946520595 conn(0x7f120407a700 msgr2=0x7f120407ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:06.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 -- 172.21.15.99:0/946520595 shutdown_connections 2024-05-14T17:01:06.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.845+0000 7f1208db9700 1 -- 172.21.15.99:0/946520595 wait complete. 2024-05-14T17:01:06.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 Processor -- start 2024-05-14T17:01:06.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 -- start start 2024-05-14T17:01:06.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204103600 0x7f120419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 0x7f120419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12041095e0 0x7f1204199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12041136e0 con 0x7f1204103fc0 2024-05-14T17:01:07.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1204113560 con 0x7f12041095e0 2024-05-14T17:01:07.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1208db9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1204113860 con 0x7f1204103600 2024-05-14T17:01:07.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 0x7f120419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 0x7f120419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35120/0 (socket says 172.21.15.99:35120) 2024-05-14T17:01:07.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 -- 172.21.15.99:0/797109306 learned_addr learned my addr 172.21.15.99:0/797109306 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:07.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f120259c700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204103600 0x7f120419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f1202d9d700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12041095e0 0x7f1204199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 -- 172.21.15.99:0/797109306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204103600 msgr2=0x7f120419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204103600 0x7f120419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 -- 172.21.15.99:0/797109306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12041095e0 msgr2=0x7f1204199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12041095e0 0x7f1204199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.846+0000 7f11fbfff700 1 -- 172.21.15.99:0/797109306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120419a080 con 0x7f1204103fc0 2024-05-14T17:01:07.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.847+0000 7f11fbfff700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 0x7f120419fca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00b580 tx=0x7f11ec0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:07.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.847+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec027070 con 0x7f1204103fc0 2024-05-14T17:01:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.847+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120419a310 con 0x7f1204103fc0 2024-05-14T17:01:07.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.847+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec004110 con 0x7f1204103fc0 2024-05-14T17:01:07.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.847+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00f460 con 0x7f1204103fc0 2024-05-14T17:01:07.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.847+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12041a6630 con 0x7f1204103fc0 2024-05-14T17:01:07.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.848+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120406a230 con 0x7f1204103fc0 2024-05-14T17:01:07.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.849+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f11ec02a070 con 0x7f1204103fc0 2024-05-14T17:01:07.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.850+0000 7f11fb7fe700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f11f00b2910 0x7f11f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.850+0000 7f120259c700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f11f00b2910 0x7f11f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.850+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f11ec0e3780 con 0x7f1204103fc0 2024-05-14T17:01:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.850+0000 7f120259c700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f11f00b2910 0x7f11f00b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005fd0 tx=0x7f11f400af00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:07.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:06.853+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11ec031070 con 0x7f1204103fc0 2024-05-14T17:01:07.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.058+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f120402e560 con 0x7f1204103fc0 2024-05-14T17:01:07.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.059+0000 7f11fb7fe700 1 -- 172.21.15.99:0/797109306 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22695 (secure 0 0 0) 0x7f11ec013070 con 0x7f1204103fc0 2024-05-14T17:01:07.263 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:01:07.271 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:01:04.227948+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:01:07.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f11f00b2910 msgr2=0x7f11f00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f11f00b2910 0x7f11f00b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11f4005fd0 tx=0x7f11f400af00 comp rx=0 tx=0).stop 2024-05-14T17:01:07.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 msgr2=0x7f120419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 0x7f120419fca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00b580 tx=0x7f11ec0037c0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 shutdown_connections 2024-05-14T17:01:07.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f11f00b2910 0x7f11f00b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12041095e0 0x7f1204199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204103600 0x7f120419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 --2- 172.21.15.99:0/797109306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1204103fc0 0x7f120419fca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 >> 172.21.15.99:0/797109306 conn(0x7f120407a700 msgr2=0x7f1204102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:07.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 shutdown_connections 2024-05-14T17:01:07.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:07.062+0000 7f1208db9700 1 -- 172.21.15.99:0/797109306 wait complete. 2024-05-14T17:01:07.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:07 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/797109306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:01:07.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:07 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/797109306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:01:07.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:07 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/797109306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:01:07.743 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-14T17:01:07.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:01:07.998 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:08.441 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:08 smithi099 ceph-mon[31851]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:08.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:08 smithi193 ceph-mon[35962]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:08.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:08 smithi150 ceph-mon[36204]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:09.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 -- 172.21.15.99:0/117623036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888103850 msgr2=0x7f8888103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 --2- 172.21.15.99:0/117623036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888103850 0x7f8888103c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f887c0099c0 tx=0x7f887c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 -- 172.21.15.99:0/117623036 shutdown_connections 2024-05-14T17:01:09.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 --2- 172.21.15.99:0/117623036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888108920 0x7f88881104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 --2- 172.21.15.99:0/117623036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888103850 0x7f8888103c30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 --2- 172.21.15.99:0/117623036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8888104200 0x7f88881081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 -- 172.21.15.99:0/117623036 >> 172.21.15.99:0/117623036 conn(0x7f888807a620 msgr2=0x7f888807aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:09.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 -- 172.21.15.99:0/117623036 shutdown_connections 2024-05-14T17:01:09.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.550+0000 7f888ec7d700 1 -- 172.21.15.99:0/117623036 wait complete. 2024-05-14T17:01:09.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 Processor -- start 2024-05-14T17:01:09.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 -- start start 2024-05-14T17:01:09.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8888103850 0x7f888819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 0x7f888819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888108920 0x7f88881999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8888113760 con 0x7f8888104200 2024-05-14T17:01:09.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88881135e0 con 0x7f8888108920 2024-05-14T17:01:09.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888ec7d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88881138e0 con 0x7f8888103850 2024-05-14T17:01:09.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f8887fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 0x7f888819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f8887fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 0x7f888819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35150/0 (socket says 172.21.15.99:35150) 2024-05-14T17:01:09.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f8887fff700 1 -- 172.21.15.99:0/3083493361 learned_addr learned my addr 172.21.15.99:0/3083493361 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:09.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.551+0000 7f888d21a700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888108920 0x7f88881999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f888ca19700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8888103850 0x7f888819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8887fff700 1 -- 172.21.15.99:0/3083493361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8888103850 msgr2=0x7f888819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8887fff700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8888103850 0x7f888819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8887fff700 1 -- 172.21.15.99:0/3083493361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888108920 msgr2=0x7f88881999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8887fff700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888108920 0x7f88881999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8887fff700 1 -- 172.21.15.99:0/3083493361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888819a100 con 0x7f8888104200 2024-05-14T17:01:09.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8887fff700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 0x7f888819fd20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f887400b530 tx=0x7f887400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:09.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874015020 con 0x7f8888104200 2024-05-14T17:01:09.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8874011ce0 con 0x7f8888104200 2024-05-14T17:01:09.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888819a3f0 con 0x7f8888104200 2024-05-14T17:01:09.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874010e40 con 0x7f8888104200 2024-05-14T17:01:09.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.552+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a66b0 con 0x7f8888104200 2024-05-14T17:01:09.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.553+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888810bb60 con 0x7f8888104200 2024-05-14T17:01:09.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.554+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8874018020 con 0x7f8888104200 2024-05-14T17:01:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.555+0000 7f8885ffb700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f88780b2990 0x7f88780b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:09.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.555+0000 7f888ca19700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f88780b2990 0x7f88780b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:09.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.555+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f88740e1a20 con 0x7f8888104200 2024-05-14T17:01:09.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.556+0000 7f888ca19700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f88780b2990 0x7f88780b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f887c000c00 tx=0x7f887c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:10.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.560+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f887401f020 con 0x7f8888104200 2024-05-14T17:01:10.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.764+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f888810b540 con 0x7f8888104200 2024-05-14T17:01:10.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.766+0000 7f8885ffb700 1 -- 172.21.15.99:0/3083493361 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22695 (secure 0 0 0) 0x7f88740a9700 con 0x7f8888104200 2024-05-14T17:01:10.022 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:01:10.029 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:01:04.227948+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:01:10.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f88780b2990 msgr2=0x7f88780b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f88780b2990 0x7f88780b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f887c000c00 tx=0x7f887c005fb0 comp rx=0 tx=0).stop 2024-05-14T17:01:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 msgr2=0x7f888819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:10.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 0x7f888819fd20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f887400b530 tx=0x7f887400b840 comp rx=0 tx=0).stop 2024-05-14T17:01:10.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 shutdown_connections 2024-05-14T17:01:10.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f88780b2990 0x7f88780b4e50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:10.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888108920 0x7f88881999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:10.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8888103850 0x7f888819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 --2- 172.21.15.99:0/3083493361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8888104200 0x7f888819fd20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:10.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 >> 172.21.15.99:0/3083493361 conn(0x7f888807a620 msgr2=0x7f8888102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:10.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.768+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 shutdown_connections 2024-05-14T17:01:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:09.769+0000 7f888ec7d700 1 -- 172.21.15.99:0/3083493361 wait complete. 2024-05-14T17:01:10.403 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:10 smithi099 ceph-mon[31851]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:10.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:10 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3083493361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:01:10.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-14T17:01:10.426 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-14T17:01:10.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-14T17:01:10.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-14T17:01:10.446 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-14T17:01:10.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-14T17:01:10.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-14T17:01:10.465 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-14T17:01:10.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-14T17:01:10.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-14T17:01:10.513 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-14T17:01:10.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-14T17:01:10.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:10 smithi193 ceph-mon[35962]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:10.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:10 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3083493361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:01:10.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:10 smithi150 ceph-mon[36204]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:10.717 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:10 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3083493361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:01:11.200 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.876 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.884 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.892 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.899 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.907 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.915 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.970 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:11.977 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:12.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:12 smithi099 ceph-mon[31851]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 130 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:12.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:12 smithi193 ceph-mon[35962]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 130 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:12.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:12 smithi150 ceph-mon[36204]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 130 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:12.805 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:14.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:14 smithi099 ceph-mon[31851]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 113 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:14.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:14 smithi193 ceph-mon[35962]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 113 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:14.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:14 smithi150 ceph-mon[36204]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 113 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T17:01:16.461 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:16 smithi099 ceph-mon[31851]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 102 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T17:01:16.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:16 smithi193 ceph-mon[35962]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 102 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T17:01:16.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:16 smithi150 ceph-mon[36204]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 102 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T17:01:17.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 -- 172.21.15.99:0/1785069545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac006bc90 msgr2=0x7f1ac006c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:17.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 --2- 172.21.15.99:0/1785069545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac006bc90 0x7f1ac006c110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009990 tx=0x7f1ab4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 -- 172.21.15.99:0/1785069545 shutdown_connections 2024-05-14T17:01:17.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 --2- 172.21.15.99:0/1785069545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac006c650 0x7f1ac01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 --2- 172.21.15.99:0/1785069545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac01095b0 0x7f1ac0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 --2- 172.21.15.99:0/1785069545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac006bc90 0x7f1ac006c110 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:17.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 -- 172.21.15.99:0/1785069545 >> 172.21.15.99:0/1785069545 conn(0x7f1ac0079410 msgr2=0x7f1ac0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:17.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 -- 172.21.15.99:0/1785069545 shutdown_connections 2024-05-14T17:01:17.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.760+0000 7f1ac4989700 1 -- 172.21.15.99:0/1785069545 wait complete. 2024-05-14T17:01:17.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 Processor -- start 2024-05-14T17:01:17.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 -- start start 2024-05-14T17:01:17.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac006bc90 0x7f1ac019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:17.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac006c650 0x7f1ac019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:17.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac01095b0 0x7f1ac01a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:17.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0113760 con 0x7f1ac01095b0 2024-05-14T17:01:17.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01135e0 con 0x7f1ac006c650 2024-05-14T17:01:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1ac4989700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01138e0 con 0x7f1ac006bc90 2024-05-14T17:01:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1abed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac01095b0 0x7f1ac01a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1abed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac01095b0 0x7f1ac01a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54126/0 (socket says 172.21.15.99:54126) 2024-05-14T17:01:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1abed9d700 1 -- 172.21.15.99:0/2960537751 learned_addr learned my addr 172.21.15.99:0/2960537751 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1abe59c700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac006bc90 0x7f1ac019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:17.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.761+0000 7f1abdd9b700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac006c650 0x7f1ac019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:17.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abed9d700 1 -- 172.21.15.99:0/2960537751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac006bc90 msgr2=0x7f1ac019bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:18.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abed9d700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac006bc90 0x7f1ac019bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:18.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abed9d700 1 -- 172.21.15.99:0/2960537751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac006c650 msgr2=0x7f1ac019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:18.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abed9d700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac006c650 0x7f1ac019c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:18.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abed9d700 1 -- 172.21.15.99:0/2960537751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac01a0f30 con 0x7f1ac01095b0 2024-05-14T17:01:18.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abe59c700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac006bc90 0x7f1ac019bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1abed9d700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac01095b0 0x7f1ac01a07f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8007ee0 tx=0x7f1ab800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:18.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8004bb0 con 0x7f1ac01095b0 2024-05-14T17:01:18.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab8004d50 con 0x7f1ac01095b0 2024-05-14T17:01:18.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac01a1220 con 0x7f1ac01095b0 2024-05-14T17:01:18.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8005920 con 0x7f1ac01095b0 2024-05-14T17:01:18.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.762+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac01a8aa0 con 0x7f1ac01095b0 2024-05-14T17:01:18.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.763+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ac0101880 con 0x7f1ac01095b0 2024-05-14T17:01:18.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.765+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1ab801a020 con 0x7f1ac01095b0 2024-05-14T17:01:18.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.765+0000 7f1aaf7fe700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1aa80b2910 0x7f1aa80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:18.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.765+0000 7f1abe59c700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1aa80b2910 0x7f1aa80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:18.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.765+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f1ab80e2480 con 0x7f1ac01095b0 2024-05-14T17:01:18.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.766+0000 7f1aaf7fe700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] conn(0x7f1aa80bc450 0x7f1aa80be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:18.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.766+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 --> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ab8023e70 con 0x7f1aa80bc450 2024-05-14T17:01:18.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.766+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f1ab80e28d0 con 0x7f1ac01095b0 2024-05-14T17:01:18.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.766+0000 7f1abdd9b700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] conn(0x7f1aa80bc450 0x7f1aa80be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:18.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.766+0000 7f1abe59c700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1aa80b2910 0x7f1aa80b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009de0 tx=0x7f1ab0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:18.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.766+0000 7f1abdd9b700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] conn(0x7f1aa80bc450 0x7f1aa80be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:18.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.767+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== osd.3 v2:172.21.15.99:6810/429217969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ab8023e70 con 0x7f1aa80bc450 2024-05-14T17:01:18.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.783+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 --> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1ac0104c70 con 0x7f1aa80bc450 2024-05-14T17:01:18.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1aaf7fe700 1 -- 172.21.15.99:0/2960537751 <== osd.3 v2:172.21.15.99:6810/429217969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1ac0104c70 con 0x7f1aa80bc450 2024-05-14T17:01:18.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] conn(0x7f1aa80bc450 msgr2=0x7f1aa80be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] conn(0x7f1aa80bc450 0x7f1aa80be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:18.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1aa80b2910 msgr2=0x7f1aa80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:18.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f1aa80b2910 0x7f1aa80b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009de0 tx=0x7f1ab0009190 comp rx=0 tx=0).stop 2024-05-14T17:01:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac01095b0 msgr2=0x7f1ac01a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 --2- 172.21.15.99:0/2960537751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ac01095b0 0x7f1ac01a07f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8007ee0 tx=0x7f1ab800d3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:18.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1abed9d700 1 -- 172.21.15.99:0/2960537751 reap_dead start 2024-05-14T17:01:18.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 shutdown_connections 2024-05-14T17:01:18.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 >> 172.21.15.99:0/2960537751 conn(0x7f1ac0079410 msgr2=0x7f1ac0100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:18.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 shutdown_connections 2024-05-14T17:01:18.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:17.784+0000 7f1ac4989700 1 -- 172.21.15.99:0/2960537751 wait complete. 2024-05-14T17:01:18.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:18 smithi099 ceph-mon[31851]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 87 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T17:01:18.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:18 smithi193 ceph-mon[35962]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 87 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T17:01:18.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:18 smithi150 ceph-mon[36204]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 87 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T17:01:20.486 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:20 smithi099 ceph-mon[31851]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 84 B/s, 0 objects/s recovering 2024-05-14T17:01:20.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:20 smithi193 ceph-mon[35962]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 84 B/s, 0 objects/s recovering 2024-05-14T17:01:20.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:20 smithi150 ceph-mon[36204]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 84 B/s, 0 objects/s recovering 2024-05-14T17:01:21.153 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:21.192 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:21.584 INFO:teuthology.orchestra.run.smithi099.stdout:111669149765 2024-05-14T17:01:21.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-14T17:01:22.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:22 smithi099 ceph-mon[31851]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 84 B/s, 0 objects/s recovering 2024-05-14T17:01:22.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:22 smithi193 ceph-mon[35962]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 84 B/s, 0 objects/s recovering 2024-05-14T17:01:22.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:22 smithi150 ceph-mon[36204]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 84 B/s, 0 objects/s recovering 2024-05-14T17:01:23.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 -- 172.21.15.99:0/1793893673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb006bab0 msgr2=0x7fdbb006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1793893673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb006bab0 0x7fdbb006bf30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009990 tx=0x7fdba4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 -- 172.21.15.99:0/1793893673 shutdown_connections 2024-05-14T17:01:23.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1793893673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb006c470 0x7fdbb01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1793893673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb01095a0 0x7fdbb0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1793893673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb006bab0 0x7fdbb006bf30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 -- 172.21.15.99:0/1793893673 >> 172.21.15.99:0/1793893673 conn(0x7fdbb00793b0 msgr2=0x7fdbb00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 -- 172.21.15.99:0/1793893673 shutdown_connections 2024-05-14T17:01:23.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.234+0000 7fdbb792a700 1 -- 172.21.15.99:0/1793893673 wait complete. 2024-05-14T17:01:23.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 Processor -- start 2024-05-14T17:01:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 -- start start 2024-05-14T17:01:23.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb006bab0 0x7fdbb019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:23.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb006c470 0x7fdbb019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:23.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb01095a0 0x7fdbb0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:23.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb01136d0 con 0x7fdbb01095a0 2024-05-14T17:01:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0113550 con 0x7fdbb006bab0 2024-05-14T17:01:23.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb792a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0113850 con 0x7fdbb006c470 2024-05-14T17:01:23.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb01095a0 0x7fdbb0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb01095a0 0x7fdbb0199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54134/0 (socket says 172.21.15.99:54134) 2024-05-14T17:01:23.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 -- 172.21.15.99:0/1540293919 learned_addr learned my addr 172.21.15.99:0/1540293919 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:23.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb4ec5700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb006c470 0x7fdbb019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:23.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 -- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb006c470 msgr2=0x7fdbb019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb006c470 0x7fdbb019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 -- 172.21.15.99:0/1540293919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb006bab0 msgr2=0x7fdbb019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb56c6700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb006bab0 0x7fdbb019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:23.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb006bab0 0x7fdbb019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb5ec7700 1 -- 172.21.15.99:0/1540293919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb019a210 con 0x7fdbb01095a0 2024-05-14T17:01:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.235+0000 7fdbb4ec5700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb006c470 0x7fdbb019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbb5ec7700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb01095a0 0x7fdbb0199ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d980 tx=0x7fdba000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0017070 con 0x7fdbb01095a0 2024-05-14T17:01:23.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba0009df0 con 0x7fdbb01095a0 2024-05-14T17:01:23.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb019a500 con 0x7fdbb01095a0 2024-05-14T17:01:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba00053a0 con 0x7fdbb01095a0 2024-05-14T17:01:23.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a67c0 con 0x7fdbb01095a0 2024-05-14T17:01:23.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.236+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdbb0063c50 con 0x7fdbb01095a0 2024-05-14T17:01:23.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.238+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdba0004750 con 0x7fdbb01095a0 2024-05-14T17:01:23.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbae7fc700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdb980b2990 0x7fdb980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbb56c6700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdb980b2990 0x7fdb980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:23.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fdba00e2880 con 0x7fdbb01095a0 2024-05-14T17:01:23.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbae7fc700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] conn(0x7fdb980bc4d0 0x7fdb980be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:23.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 --> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdba001ae70 con 0x7fdb980bc4d0 2024-05-14T17:01:23.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fdba00e2cd0 con 0x7fdbb01095a0 2024-05-14T17:01:23.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbb56c6700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdb980b2990 0x7fdb980b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdba80060b0 tx=0x7fdba80085c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:23.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.239+0000 7fdbb4ec5700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] conn(0x7fdb980bc4d0 0x7fdb980be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.240+0000 7fdbb4ec5700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] conn(0x7fdb980bc4d0 0x7fdb980be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:23.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.241+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== osd.5 v2:172.21.15.193:6808/2563526327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdba001ae70 con 0x7fdb980bc4d0 2024-05-14T17:01:23.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.258+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 --> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdbb0064050 con 0x7fdb980bc4d0 2024-05-14T17:01:23.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbae7fc700 1 -- 172.21.15.99:0/1540293919 <== osd.5 v2:172.21.15.193:6808/2563526327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdbb0064050 con 0x7fdb980bc4d0 2024-05-14T17:01:23.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] conn(0x7fdb980bc4d0 msgr2=0x7fdb980be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] conn(0x7fdb980bc4d0 0x7fdb980be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdb980b2990 msgr2=0x7fdb980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdb980b2990 0x7fdb980b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdba80060b0 tx=0x7fdba80085c0 comp rx=0 tx=0).stop 2024-05-14T17:01:23.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb01095a0 msgr2=0x7fdbb0199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:23.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 --2- 172.21.15.99:0/1540293919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbb01095a0 0x7fdbb0199ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d980 tx=0x7fdba000dd40 comp rx=0 tx=0).stop 2024-05-14T17:01:23.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb5ec7700 1 -- 172.21.15.99:0/1540293919 reap_dead start 2024-05-14T17:01:23.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 shutdown_connections 2024-05-14T17:01:23.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 >> 172.21.15.99:0/1540293919 conn(0x7fdbb00793b0 msgr2=0x7fdbb0101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:23.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 shutdown_connections 2024-05-14T17:01:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.259+0000 7fdbb792a700 1 -- 172.21.15.99:0/1540293919 wait complete. 2024-05-14T17:01:24.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 -- 172.21.15.99:0/583498964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f41095b0 msgr2=0x7f30f4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 --2- 172.21.15.99:0/583498964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f41095b0 0x7f30f4109990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009990 tx=0x7f30e4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 -- 172.21.15.99:0/583498964 shutdown_connections 2024-05-14T17:01:24.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 --2- 172.21.15.99:0/583498964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30f406bbe0 0x7f30f41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 --2- 172.21.15.99:0/583498964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f41095b0 0x7f30f4109990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 --2- 172.21.15.99:0/583498964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f406b220 0x7f30f406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 -- 172.21.15.99:0/583498964 >> 172.21.15.99:0/583498964 conn(0x7f30f40793f0 msgr2=0x7f30f4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:24.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 -- 172.21.15.99:0/583498964 shutdown_connections 2024-05-14T17:01:24.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.746+0000 7f30fa231700 1 -- 172.21.15.99:0/583498964 wait complete. 2024-05-14T17:01:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.747+0000 7f30fa231700 1 Processor -- start 2024-05-14T17:01:24.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.747+0000 7f30fa231700 1 -- start start 2024-05-14T17:01:24.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.747+0000 7f30fa231700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30f406b220 0x7f30f419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f406bbe0 0x7f30f419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f41095b0 0x7f30f4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30f4113740 con 0x7f30f406bbe0 2024-05-14T17:01:24.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f30f41135c0 con 0x7f30f41095b0 2024-05-14T17:01:24.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30f41138c0 con 0x7f30f406b220 2024-05-14T17:01:24.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f406bbe0 0x7f30f419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f406bbe0 0x7f30f419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54148/0 (socket says 172.21.15.99:54148) 2024-05-14T17:01:24.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 -- 172.21.15.99:0/2967885862 learned_addr learned my addr 172.21.15.99:0/2967885862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:24.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f37fe700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30f406b220 0x7f30f419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f3fff700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f41095b0 0x7f30f4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 -- 172.21.15.99:0/2967885862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30f406b220 msgr2=0x7f30f419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30f406b220 0x7f30f419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 -- 172.21.15.99:0/2967885862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f41095b0 msgr2=0x7f30f4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f41095b0 0x7f30f4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 -- 172.21.15.99:0/2967885862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f419a1e0 con 0x7f30f406bbe0 2024-05-14T17:01:24.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f2ffd700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f406bbe0 0x7f30f419fe00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b530 tx=0x7f30e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:24.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8015020 con 0x7f30f406bbe0 2024-05-14T17:01:24.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e8011ce0 con 0x7f30f406bbe0 2024-05-14T17:01:24.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8010e40 con 0x7f30f406bbe0 2024-05-14T17:01:24.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f419a4d0 con 0x7f30f406bbe0 2024-05-14T17:01:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.748+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f41a6790 con 0x7f30f406bbe0 2024-05-14T17:01:24.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.749+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30f4063cd0 con 0x7f30f406bbe0 2024-05-14T17:01:24.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.751+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f30e8018020 con 0x7f30f406bbe0 2024-05-14T17:01:24.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.751+0000 7f30f0ff9700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f30dc0b2910 0x7f30dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.751+0000 7f30f37fe700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f30dc0b2910 0x7f30dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f30e80e2230 con 0x7f30f406bbe0 2024-05-14T17:01:24.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f0ff9700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] conn(0x7f30dc0bc450 0x7f30dc0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:24.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 --> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30e801be50 con 0x7f30dc0bc450 2024-05-14T17:01:24.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f30e80e2680 con 0x7f30f406bbe0 2024-05-14T17:01:24.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f3fff700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] conn(0x7f30dc0bc450 0x7f30dc0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:24.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f37fe700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f30dc0b2910 0x7f30dc0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b5c0 tx=0x7f30e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:24.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.752+0000 7f30f3fff700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] conn(0x7f30dc0bc450 0x7f30dc0be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:24.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.753+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== osd.6 v2:172.21.15.99:6818/3623238310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30e801be50 con 0x7f30dc0bc450 2024-05-14T17:01:24.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 --> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30f4064150 con 0x7f30dc0bc450 2024-05-14T17:01:24.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30f0ff9700 1 -- 172.21.15.99:0/2967885862 <== osd.6 v2:172.21.15.99:6818/3623238310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30f4064150 con 0x7f30dc0bc450 2024-05-14T17:01:24.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] conn(0x7f30dc0bc450 msgr2=0x7f30dc0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] conn(0x7f30dc0bc450 0x7f30dc0be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f30dc0b2910 msgr2=0x7f30dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f30dc0b2910 0x7f30dc0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b5c0 tx=0x7f30e4005960 comp rx=0 tx=0).stop 2024-05-14T17:01:24.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f406bbe0 msgr2=0x7f30f419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 --2- 172.21.15.99:0/2967885862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30f406bbe0 0x7f30f419fe00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b530 tx=0x7f30e800b840 comp rx=0 tx=0).stop 2024-05-14T17:01:24.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30f3fff700 1 -- 172.21.15.99:0/2967885862 reap_dead start 2024-05-14T17:01:24.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 shutdown_connections 2024-05-14T17:01:24.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.770+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 >> 172.21.15.99:0/2967885862 conn(0x7f30f40793f0 msgr2=0x7f30f4101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:24.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.771+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 shutdown_connections 2024-05-14T17:01:24.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:23.771+0000 7f30fa231700 1 -- 172.21.15.99:0/2967885862 wait complete. 2024-05-14T17:01:24.900 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:24 smithi099 ceph-mon[31851]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:24.922 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:24 smithi150 ceph-mon[36204]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:24.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:24 smithi193 ceph-mon[35962]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:24.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 -- 172.21.15.99:0/2156948588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 msgr2=0x7f9af4109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:24.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 --2- 172.21.15.99:0/2156948588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af4109980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009990 tx=0x7f9ae4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 -- 172.21.15.99:0/2156948588 shutdown_connections 2024-05-14T17:01:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 --2- 172.21.15.99:0/2156948588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af406c470 0x7f9af41103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 --2- 172.21.15.99:0/2156948588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af4109980 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 --2- 172.21.15.99:0/2156948588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af406bab0 0x7f9af406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:24.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 -- 172.21.15.99:0/2156948588 >> 172.21.15.99:0/2156948588 conn(0x7f9af40793b0 msgr2=0x7f9af40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:24.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 -- 172.21.15.99:0/2156948588 shutdown_connections 2024-05-14T17:01:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.948+0000 7f9afa46b700 1 -- 172.21.15.99:0/2156948588 wait complete. 2024-05-14T17:01:25.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 Processor -- start 2024-05-14T17:01:25.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 -- start start 2024-05-14T17:01:25.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af406bab0 0x7f9af4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af406c470 0x7f9af419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41137b0 con 0x7f9af41095a0 2024-05-14T17:01:25.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4113630 con 0x7f9af406c470 2024-05-14T17:01:25.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9afa46b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4113930 con 0x7f9af406bab0 2024-05-14T17:01:25.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38488/0 (socket says 172.21.15.99:38488) 2024-05-14T17:01:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 -- 172.21.15.99:0/606989127 learned_addr learned my addr 172.21.15.99:0/606989127 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af3fff700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af406bab0 0x7f9af4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af37fe700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af406c470 0x7f9af419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 -- 172.21.15.99:0/606989127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af406bab0 msgr2=0x7f9af4199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af406bab0 0x7f9af4199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 -- 172.21.15.99:0/606989127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af406c470 msgr2=0x7f9af419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:25.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af406c470 0x7f9af419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.949+0000 7f9af8a08700 1 -- 172.21.15.99:0/606989127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af419ed30 con 0x7f9af41095a0 2024-05-14T17:01:25.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9af37fe700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af406c470 0x7f9af419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9af8a08700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af419e5f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b970 tx=0x7f9aec00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:25.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec019070 con 0x7f9af41095a0 2024-05-14T17:01:25.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af419f020 con 0x7f9af41095a0 2024-05-14T17:01:25.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec014440 con 0x7f9af41095a0 2024-05-14T17:01:25.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ce60 con 0x7f9af41095a0 2024-05-14T17:01:25.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.950+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a68a0 con 0x7f9af41095a0 2024-05-14T17:01:25.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.951+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9af4063cd0 con 0x7f9af41095a0 2024-05-14T17:01:25.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.952+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9aec01c070 con 0x7f9af41095a0 2024-05-14T17:01:25.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af17fa700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9adc0b2910 0x7f9adc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af3fff700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9adc0b2910 0x7f9adc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f9aec0e1f50 con 0x7f9af41095a0 2024-05-14T17:01:25.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af17fa700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] conn(0x7f9adc0bc450 0x7f9adc0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 --> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9aec017e70 con 0x7f9adc0bc450 2024-05-14T17:01:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9aec0e23a0 con 0x7f9af41095a0 2024-05-14T17:01:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af37fe700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] conn(0x7f9adc0bc450 0x7f9adc0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af3fff700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9adc0b2910 0x7f9adc0b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4006010 tx=0x7f9ae401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:25.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.953+0000 7f9af37fe700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] conn(0x7f9adc0bc450 0x7f9adc0be8f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.954+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== osd.9 v2:172.21.15.99:6826/3111395354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9aec017e70 con 0x7f9adc0bc450 2024-05-14T17:01:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.970+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 --> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9af4064150 con 0x7f9adc0bc450 2024-05-14T17:01:25.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9af17fa700 1 -- 172.21.15.99:0/606989127 <== osd.9 v2:172.21.15.99:6826/3111395354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9af4064150 con 0x7f9adc0bc450 2024-05-14T17:01:25.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] conn(0x7f9adc0bc450 msgr2=0x7f9adc0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:25.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] conn(0x7f9adc0bc450 0x7f9adc0be8f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9adc0b2910 msgr2=0x7f9adc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:25.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9adc0b2910 0x7f9adc0b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4006010 tx=0x7f9ae401a040 comp rx=0 tx=0).stop 2024-05-14T17:01:25.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 msgr2=0x7f9af419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:25.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 --2- 172.21.15.99:0/606989127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9af41095a0 0x7f9af419e5f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b970 tx=0x7f9aec00bd30 comp rx=0 tx=0).stop 2024-05-14T17:01:25.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9af8a08700 1 -- 172.21.15.99:0/606989127 reap_dead start 2024-05-14T17:01:25.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 shutdown_connections 2024-05-14T17:01:25.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 >> 172.21.15.99:0/606989127 conn(0x7f9af40793b0 msgr2=0x7f9af4101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:25.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 shutdown_connections 2024-05-14T17:01:25.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:24.971+0000 7f9afa46b700 1 -- 172.21.15.99:0/606989127 wait complete. 2024-05-14T17:01:25.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3856307046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78108500 msgr2=0x7fbe781088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:25.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3856307046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78108500 0x7fbe781088e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000b0a0 tx=0x7fbe7000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3856307046 shutdown_connections 2024-05-14T17:01:25.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3856307046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe78103f90 0x7fbe78110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3856307046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78108500 0x7fbe781088e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3856307046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe781035f0 0x7fbe78103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3856307046 >> 172.21.15.99:0/3856307046 conn(0x7fbe7807a6a0 msgr2=0x7fbe7807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:25.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3856307046 shutdown_connections 2024-05-14T17:01:25.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.300+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3856307046 wait complete. 2024-05-14T17:01:25.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 Processor -- start 2024-05-14T17:01:25.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 -- start start 2024-05-14T17:01:25.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe781035f0 0x7fbe7819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe78103f90 0x7fbe7819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe78108500 0x7fbe781a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:25.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78113790 con 0x7fbe781035f0 2024-05-14T17:01:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78113610 con 0x7fbe78103f90 2024-05-14T17:01:25.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7f45d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78113910 con 0x7fbe78108500 2024-05-14T17:01:25.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7d1f9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe781035f0 0x7fbe7819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:25.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7d1f9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe781035f0 0x7fbe7819be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38520/0 (socket says 172.21.15.99:38520) 2024-05-14T17:01:25.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7d1f9700 1 -- 172.21.15.99:0/3145196351 learned_addr learned my addr 172.21.15.99:0/3145196351 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.301+0000 7fbe7c9f8700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe78103f90 0x7fbe7819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:26.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d9fa700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe78108500 0x7fbe781a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d1f9700 1 -- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe78108500 msgr2=0x7fbe781a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:26.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d1f9700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe78108500 0x7fbe781a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:26.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d1f9700 1 -- 172.21.15.99:0/3145196351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe78103f90 msgr2=0x7fbe7819c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:26.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d1f9700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe78103f90 0x7fbe7819c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:26.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d1f9700 1 -- 172.21.15.99:0/3145196351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe781a0e40 con 0x7fbe781035f0 2024-05-14T17:01:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d9fa700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe78108500 0x7fbe781a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:26.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7d1f9700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe781035f0 0x7fbe7819be30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000d010 tx=0x7fbe70004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:26.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70021070 con 0x7fbe781035f0 2024-05-14T17:01:26.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe781a10d0 con 0x7fbe781035f0 2024-05-14T17:01:26.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe70007e30 con 0x7fbe781035f0 2024-05-14T17:01:26.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004670 con 0x7fbe781035f0 2024-05-14T17:01:26.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.302+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe781a89b0 con 0x7fbe781035f0 2024-05-14T17:01:26.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.303+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe78101ec0 con 0x7fbe781035f0 2024-05-14T17:01:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.304+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbe70007980 con 0x7fbe781035f0 2024-05-14T17:01:26.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe767fc700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe600b2910 0x7fbe600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:26.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe7c9f8700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe600b2910 0x7fbe600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:26.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fbe7000f020 con 0x7fbe781035f0 2024-05-14T17:01:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe767fc700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] conn(0x7fbe600bc450 0x7fbe600be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:26.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 --> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe600befc0 con 0x7fbe600bc450 2024-05-14T17:01:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fbe700e3e90 con 0x7fbe781035f0 2024-05-14T17:01:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.305+0000 7fbe7c9f8700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe600b2910 0x7fbe600b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe78109c00 tx=0x7fbe6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:26.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.306+0000 7fbe7d9fa700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] conn(0x7fbe600bc450 0x7fbe600be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:26.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.306+0000 7fbe7d9fa700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] conn(0x7fbe600bc450 0x7fbe600be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:26.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.307+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== osd.8 v2:172.21.15.193:6816/2890842755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe600befc0 con 0x7fbe600bc450 2024-05-14T17:01:26.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.323+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 --> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe78104cb0 con 0x7fbe600bc450 2024-05-14T17:01:26.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe767fc700 1 -- 172.21.15.99:0/3145196351 <== osd.8 v2:172.21.15.193:6816/2890842755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe78104cb0 con 0x7fbe600bc450 2024-05-14T17:01:26.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] conn(0x7fbe600bc450 msgr2=0x7fbe600be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:26.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] conn(0x7fbe600bc450 0x7fbe600be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:26.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe600b2910 msgr2=0x7fbe600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:26.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbe600b2910 0x7fbe600b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe78109c00 tx=0x7fbe6c009260 comp rx=0 tx=0).stop 2024-05-14T17:01:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe781035f0 msgr2=0x7fbe7819be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:26.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 --2- 172.21.15.99:0/3145196351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe781035f0 0x7fbe7819be30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000d010 tx=0x7fbe70004d00 comp rx=0 tx=0).stop 2024-05-14T17:01:26.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7d9fa700 1 -- 172.21.15.99:0/3145196351 reap_dead start 2024-05-14T17:01:26.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 shutdown_connections 2024-05-14T17:01:26.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 >> 172.21.15.99:0/3145196351 conn(0x7fbe7807a6a0 msgr2=0x7fbe78101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:26.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.324+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 shutdown_connections 2024-05-14T17:01:26.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:25.325+0000 7fbe7f45d700 1 -- 172.21.15.99:0/3145196351 wait complete. 2024-05-14T17:01:26.997 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:26 smithi150 ceph-mon[36204]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-14T17:01:27.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:26 smithi193 ceph-mon[35962]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-14T17:01:27.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 -- 172.21.15.99:0/3183209827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06bab0 msgr2=0x7f7aac06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 --2- 172.21.15.99:0/3183209827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06bab0 0x7f7aac06bf30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009990 tx=0x7f7a9c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 -- 172.21.15.99:0/3183209827 shutdown_connections 2024-05-14T17:01:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 --2- 172.21.15.99:0/3183209827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aac06c470 0x7f7aac1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 --2- 172.21.15.99:0/3183209827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac1095a0 0x7f7aac109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 --2- 172.21.15.99:0/3183209827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06bab0 0x7f7aac06bf30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 -- 172.21.15.99:0/3183209827 >> 172.21.15.99:0/3183209827 conn(0x7f7aac0793b0 msgr2=0x7f7aac0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 -- 172.21.15.99:0/3183209827 shutdown_connections 2024-05-14T17:01:27.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.286+0000 7f7ab393b700 1 -- 172.21.15.99:0/3183209827 wait complete. 2024-05-14T17:01:27.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 Processor -- start 2024-05-14T17:01:27.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 -- start start 2024-05-14T17:01:27.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aac06bab0 0x7f7aac19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06c470 0x7f7aac19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac1095a0 0x7f7aac199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac113710 con 0x7f7aac06c470 2024-05-14T17:01:27.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac113590 con 0x7f7aac06bab0 2024-05-14T17:01:27.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab393b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac113890 con 0x7f7aac1095a0 2024-05-14T17:01:27.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06c470 0x7f7aac19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06c470 0x7f7aac19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38546/0 (socket says 172.21.15.99:38546) 2024-05-14T17:01:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 -- 172.21.15.99:0/939428938 learned_addr learned my addr 172.21.15.99:0/939428938 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:27.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab1ed8700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac1095a0 0x7f7aac199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab16d7700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aac06bab0 0x7f7aac19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 -- 172.21.15.99:0/939428938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac1095a0 msgr2=0x7f7aac199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac1095a0 0x7f7aac199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 -- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aac06bab0 msgr2=0x7f7aac19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aac06bab0 0x7f7aac19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab0ed6700 1 -- 172.21.15.99:0/939428938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac19a170 con 0x7f7aac06c470 2024-05-14T17:01:27.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.287+0000 7f7ab16d7700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7aac06bab0 0x7f7aac19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:27.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.288+0000 7f7ab0ed6700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06c470 0x7f7aac19fd90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c000c00 tx=0x7f7a9c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.288+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c026070 con 0x7f7aac06c470 2024-05-14T17:01:27.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.288+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a9c003f20 con 0x7f7aac06c470 2024-05-14T17:01:27.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.288+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac19a400 con 0x7f7aac06c470 2024-05-14T17:01:27.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.288+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00fce0 con 0x7f7aac06c470 2024-05-14T17:01:27.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.288+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac1a6720 con 0x7f7aac06c470 2024-05-14T17:01:27.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.289+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7aac063c50 con 0x7f7aac06c470 2024-05-14T17:01:27.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.290+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7a9c029070 con 0x7f7aac06c470 2024-05-14T17:01:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7aa27fc700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7a980b2910 0x7f7a980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7ab16d7700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7a980b2910 0x7f7a980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f7a9c0e32f0 con 0x7f7aac06c470 2024-05-14T17:01:27.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7aa27fc700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] conn(0x7f7a980bc320 0x7f7a980be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:27.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 --> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a9c00b0b0 con 0x7f7a980bc320 2024-05-14T17:01:27.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0e3740 con 0x7f7aac06c470 2024-05-14T17:01:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.291+0000 7f7ab16d7700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7a980b2910 0x7f7a980b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7aac19ab50 tx=0x7f7aa8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:27.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.292+0000 7f7ab1ed8700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] conn(0x7f7a980bc320 0x7f7a980be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.292+0000 7f7ab1ed8700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] conn(0x7f7a980bc320 0x7f7a980be7c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.293+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== osd.4 v2:172.21.15.150:6808/1354191038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a9c00b0b0 con 0x7f7a980bc320 2024-05-14T17:01:27.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.309+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 --> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7aac0640d0 con 0x7f7a980bc320 2024-05-14T17:01:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7aa27fc700 1 -- 172.21.15.99:0/939428938 <== osd.4 v2:172.21.15.150:6808/1354191038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7aac0640d0 con 0x7f7a980bc320 2024-05-14T17:01:27.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] conn(0x7f7a980bc320 msgr2=0x7f7a980be7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab393b700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] conn(0x7f7a980bc320 0x7f7a980be7c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7a980b2910 msgr2=0x7f7a980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab393b700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f7a980b2910 0x7f7a980b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7aac19ab50 tx=0x7f7aa8009040 comp rx=0 tx=0).stop 2024-05-14T17:01:27.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06c470 msgr2=0x7f7aac19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab393b700 1 --2- 172.21.15.99:0/939428938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7aac06c470 0x7f7aac19fd90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c000c00 tx=0x7f7a9c0057c0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.310+0000 7f7ab1ed8700 1 -- 172.21.15.99:0/939428938 reap_dead start 2024-05-14T17:01:27.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.311+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 shutdown_connections 2024-05-14T17:01:27.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.311+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 >> 172.21.15.99:0/939428938 conn(0x7f7aac0793b0 msgr2=0x7f7aac101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.311+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 shutdown_connections 2024-05-14T17:01:27.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:26.311+0000 7f7ab393b700 1 -- 172.21.15.99:0/939428938 wait complete. 2024-05-14T17:01:27.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:26 smithi099 ceph-mon[31851]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-14T17:01:28.088 INFO:teuthology.orchestra.run.smithi099.stdout:261993005097 2024-05-14T17:01:28.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-14T17:01:28.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:28 smithi193 ceph-mon[35962]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:28.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:28 smithi150 ceph-mon[36204]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:28.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:28 smithi099 ceph-mon[31851]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:28.837 INFO:teuthology.orchestra.run.smithi099.stdout:163208757309 2024-05-14T17:01:28.846 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-14T17:01:29.246 INFO:teuthology.orchestra.run.smithi099.stdout:188978561080 2024-05-14T17:01:29.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-14T17:01:29.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 -- 172.21.15.99:0/3432248005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806bab0 msgr2=0x7f727806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:29.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 --2- 172.21.15.99:0/3432248005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806bab0 0x7f727806bf30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7268009990 tx=0x7f7268009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 -- 172.21.15.99:0/3432248005 shutdown_connections 2024-05-14T17:01:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 --2- 172.21.15.99:0/3432248005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f727806c470 0x7f72781103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 --2- 172.21.15.99:0/3432248005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72781095a0 0x7f7278109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 --2- 172.21.15.99:0/3432248005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806bab0 0x7f727806bf30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 -- 172.21.15.99:0/3432248005 >> 172.21.15.99:0/3432248005 conn(0x7f72780793b0 msgr2=0x7f72780797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:29.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 -- 172.21.15.99:0/3432248005 shutdown_connections 2024-05-14T17:01:29.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.439+0000 7f72806b4700 1 -- 172.21.15.99:0/3432248005 wait complete. 2024-05-14T17:01:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.440+0000 7f72806b4700 1 Processor -- start 2024-05-14T17:01:29.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.440+0000 7f72806b4700 1 -- start start 2024-05-14T17:01:29.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f72806b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f727806bab0 0x7f727819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f72806b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806c470 0x7f727819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f72806b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72781095a0 0x7f72781978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f72806b4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7278111580 con 0x7f727806c470 2024-05-14T17:01:29.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f72806b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7278111400 con 0x7f727806bab0 2024-05-14T17:01:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f72806b4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7278111700 con 0x7f72781095a0 2024-05-14T17:01:29.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806c470 0x7f727819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806c470 0x7f727819dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38564/0 (socket says 172.21.15.99:38564) 2024-05-14T17:01:29.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 -- 172.21.15.99:0/3396459847 learned_addr learned my addr 172.21.15.99:0/3396459847 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:29.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727e450700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f727806bab0 0x7f727819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:29.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727ec51700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72781095a0 0x7f72781978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:29.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 -- 172.21.15.99:0/3396459847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72781095a0 msgr2=0x7f72781978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:29.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72781095a0 0x7f72781978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 -- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f727806bab0 msgr2=0x7f727819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:29.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f727806bab0 0x7f727819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:29.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 -- 172.21.15.99:0/3396459847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7278198080 con 0x7f727806c470 2024-05-14T17:01:29.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727ec51700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72781095a0 0x7f72781978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f727dc4f700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806c470 0x7f727819dc10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7268005910 tx=0x7f72680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:29.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.441+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268027070 con 0x7f727806c470 2024-05-14T17:01:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.442+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7268003f20 con 0x7f727806c470 2024-05-14T17:01:29.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.442+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7278198310 con 0x7f727806c470 2024-05-14T17:01:29.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.442+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268022c70 con 0x7f727806c470 2024-05-14T17:01:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.442+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72781a4530 con 0x7f727806c470 2024-05-14T17:01:29.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.442+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7278063cd0 con 0x7f727806c470 2024-05-14T17:01:29.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.444+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f726802a070 con 0x7f727806c470 2024-05-14T17:01:29.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f726f7fe700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f72640b2910 0x7f72640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f727e450700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f72640b2910 0x7f72640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:29.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f72680e3f90 con 0x7f727806c470 2024-05-14T17:01:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f726f7fe700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] conn(0x7f72640bc450 0x7f72640be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 --> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7268014e50 con 0x7f72640bc450 2024-05-14T17:01:30.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f72680e43e0 con 0x7f727806c470 2024-05-14T17:01:30.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f727e450700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f72640b2910 0x7f72640b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7274005950 tx=0x7f727400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:30.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.445+0000 7f727ec51700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] conn(0x7f72640bc450 0x7f72640be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.446+0000 7f727ec51700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] conn(0x7f72640bc450 0x7f72640be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:30.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.446+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== osd.1 v2:172.21.15.150:6800/3718481884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7268014e50 con 0x7f72640bc450 2024-05-14T17:01:30.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 --> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f72780640d0 con 0x7f72640bc450 2024-05-14T17:01:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f726f7fe700 1 -- 172.21.15.99:0/3396459847 <== osd.1 v2:172.21.15.150:6800/3718481884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f72780640d0 con 0x7f72640bc450 2024-05-14T17:01:30.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] conn(0x7f72640bc450 msgr2=0x7f72640be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] conn(0x7f72640bc450 0x7f72640be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f72640b2910 msgr2=0x7f72640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f72640b2910 0x7f72640b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7274005950 tx=0x7f727400b700 comp rx=0 tx=0).stop 2024-05-14T17:01:30.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806c470 msgr2=0x7f727819dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 --2- 172.21.15.99:0/3396459847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f727806c470 0x7f727819dc10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7268005910 tx=0x7f72680047b0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f727ec51700 1 -- 172.21.15.99:0/3396459847 reap_dead start 2024-05-14T17:01:30.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 shutdown_connections 2024-05-14T17:01:30.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.463+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 >> 172.21.15.99:0/3396459847 conn(0x7f72780793b0 msgr2=0x7f7278101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:30.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.464+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 shutdown_connections 2024-05-14T17:01:30.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:29.464+0000 7f72806b4700 1 -- 172.21.15.99:0/3396459847 wait complete. 2024-05-14T17:01:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 -- 172.21.15.99:0/2783764282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41024f0 msgr2=0x7f91c41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/2783764282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41024f0 0x7f91c41103d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00b0d0 tx=0x7f91bc00b3e0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 -- 172.21.15.99:0/2783764282 shutdown_connections 2024-05-14T17:01:30.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/2783764282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41024f0 0x7f91c41103d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/2783764282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c4109520 0x7f91c4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/2783764282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c4101b30 0x7f91c4101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 -- 172.21.15.99:0/2783764282 >> 172.21.15.99:0/2783764282 conn(0x7f91c4078ab0 msgr2=0x7f91c40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:30.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 -- 172.21.15.99:0/2783764282 shutdown_connections 2024-05-14T17:01:30.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.069+0000 7f91cc3cd700 1 -- 172.21.15.99:0/2783764282 wait complete. 2024-05-14T17:01:30.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 Processor -- start 2024-05-14T17:01:30.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 -- start start 2024-05-14T17:01:30.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c4101b30 0x7f91c4102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c41024f0 0x7f91c4103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c4109520 0x7f91c4103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4113760 con 0x7f91c4101b30 2024-05-14T17:01:30.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41135e0 con 0x7f91c4109520 2024-05-14T17:01:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91cc3cd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41138e0 con 0x7f91c41024f0 2024-05-14T17:01:30.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c4101b30 0x7f91c4102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c4101b30 0x7f91c4102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38580/0 (socket says 172.21.15.99:38580) 2024-05-14T17:01:30.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 -- 172.21.15.99:0/4136908379 learned_addr learned my addr 172.21.15.99:0/4136908379 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91c9968700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c41024f0 0x7f91c4103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 -- 172.21.15.99:0/4136908379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c41024f0 msgr2=0x7f91c4103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c41024f0 0x7f91c4103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 -- 172.21.15.99:0/4136908379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c4109520 msgr2=0x7f91c4103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca96a700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c4109520 0x7f91c4103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c4109520 0x7f91c4103790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 -- 172.21.15.99:0/4136908379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c41a80e0 con 0x7f91c4101b30 2024-05-14T17:01:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.070+0000 7f91ca169700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c4101b30 0x7f91c4102d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91c000e9c0 tx=0x7f91c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.071+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000cbd0 con 0x7f91c4101b30 2024-05-14T17:01:30.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.071+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c000cd70 con 0x7f91c4101b30 2024-05-14T17:01:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.071+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c41a83d0 con 0x7f91c4101b30 2024-05-14T17:01:30.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.071+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0005610 con 0x7f91c4101b30 2024-05-14T17:01:30.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.071+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c41a8910 con 0x7f91c4101b30 2024-05-14T17:01:30.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.072+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91c4063c50 con 0x7f91c4101b30 2024-05-14T17:01:30.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.073+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f91c0004750 con 0x7f91c4101b30 2024-05-14T17:01:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91bb7fe700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f91b00b2990 0x7f91b00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91c9968700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f91b00b2990 0x7f91b00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f91c00e29e0 con 0x7f91c4101b30 2024-05-14T17:01:30.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91bb7fe700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] conn(0x7f91b00bc4d0 0x7f91b00be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 --> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91b00bf040 con 0x7f91b00bc4d0 2024-05-14T17:01:30.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f91c00e2e30 con 0x7f91c4101b30 2024-05-14T17:01:30.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91ca96a700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] conn(0x7f91b00bc4d0 0x7f91b00be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:30.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91c9968700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f91b00b2990 0x7f91b00b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91c4108c40 tx=0x7f91b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.074+0000 7f91ca96a700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] conn(0x7f91b00bc4d0 0x7f91b00be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:30.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.075+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== osd.0 v2:172.21.15.99:6802/3363601004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91b00bf040 con 0x7f91b00bc4d0 2024-05-14T17:01:30.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.092+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 --> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91c40640d0 con 0x7f91b00bc4d0 2024-05-14T17:01:30.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.092+0000 7f91bb7fe700 1 -- 172.21.15.99:0/4136908379 <== osd.0 v2:172.21.15.99:6802/3363601004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f91c40640d0 con 0x7f91b00bc4d0 2024-05-14T17:01:30.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:30 smithi150 ceph-mon[36204]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:01:30.826 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:30 smithi193 ceph-mon[35962]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:01:30.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.092+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] conn(0x7f91b00bc4d0 msgr2=0x7f91b00be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.092+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] conn(0x7f91b00bc4d0 0x7f91b00be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f91b00b2990 msgr2=0x7f91b00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f91b00b2990 0x7f91b00b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91c4108c40 tx=0x7f91b4009260 comp rx=0 tx=0).stop 2024-05-14T17:01:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c4101b30 msgr2=0x7f91c4102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 --2- 172.21.15.99:0/4136908379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c4101b30 0x7f91c4102d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91c000e9c0 tx=0x7f91c000ecd0 comp rx=0 tx=0).stop 2024-05-14T17:01:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91ca96a700 1 -- 172.21.15.99:0/4136908379 reap_dead start 2024-05-14T17:01:30.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 shutdown_connections 2024-05-14T17:01:30.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 >> 172.21.15.99:0/4136908379 conn(0x7f91c4078ab0 msgr2=0x7f91c4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 shutdown_connections 2024-05-14T17:01:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.093+0000 7f91cc3cd700 1 -- 172.21.15.99:0/4136908379 wait complete. 2024-05-14T17:01:31.028 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:30 smithi099 ceph-mon[31851]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:01:31.037 INFO:teuthology.orchestra.run.smithi099.stdout:236223201327 2024-05-14T17:01:31.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-14T17:01:31.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 -- 172.21.15.99:0/2264322988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81037f0 msgr2=0x7fdfc8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/2264322988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81037f0 0x7fdfc8103bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40099c0 tx=0x7fdfc4009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 -- 172.21.15.99:0/2264322988 shutdown_connections 2024-05-14T17:01:31.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/2264322988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc81088f0 0x7fdfc8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/2264322988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81037f0 0x7fdfc8103bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/2264322988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81041a0 0x7fdfc81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.285+0000 7fdfcfe38700 1 -- 172.21.15.99:0/2264322988 >> 172.21.15.99:0/2264322988 conn(0x7fdfc807a6a0 msgr2=0x7fdfc807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 -- 172.21.15.99:0/2264322988 shutdown_connections 2024-05-14T17:01:31.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 -- 172.21.15.99:0/2264322988 wait complete. 2024-05-14T17:01:31.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 Processor -- start 2024-05-14T17:01:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 -- start start 2024-05-14T17:01:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc81037f0 0x7fdfc819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81041a0 0x7fdfc819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81088f0 0x7fdfc81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc81136e0 con 0x7fdfc81041a0 2024-05-14T17:01:31.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8113560 con 0x7fdfc81088f0 2024-05-14T17:01:31.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.286+0000 7fdfcfe38700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8113860 con 0x7fdfc81037f0 2024-05-14T17:01:31.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81041a0 0x7fdfc819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81041a0 0x7fdfc819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38594/0 (socket says 172.21.15.99:38594) 2024-05-14T17:01:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 -- 172.21.15.99:0/1266375011 learned_addr learned my addr 172.21.15.99:0/1266375011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:31.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcdbd4700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc81037f0 0x7fdfc819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfce3d5700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81088f0 0x7fdfc81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 -- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc81037f0 msgr2=0x7fdfc819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc81037f0 0x7fdfc819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 -- 172.21.15.99:0/1266375011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81088f0 msgr2=0x7fdfc81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81088f0 0x7fdfc81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 -- 172.21.15.99:0/1266375011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc819a120 con 0x7fdfc81041a0 2024-05-14T17:01:31.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfce3d5700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfc81088f0 0x7fdfc81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:31.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcd3d3700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81041a0 0x7fdfc819fd40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800e9a0 tx=0x7fdfb800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:31.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800cbd0 con 0x7fdfc81041a0 2024-05-14T17:01:31.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc819a410 con 0x7fdfc81041a0 2024-05-14T17:01:31.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfb800cd70 con 0x7fdfc81041a0 2024-05-14T17:01:31.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8018680 con 0x7fdfc81041a0 2024-05-14T17:01:31.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.287+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc81a66d0 con 0x7fdfc81041a0 2024-05-14T17:01:31.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.288+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdfc8063c90 con 0x7fdfc81041a0 2024-05-14T17:01:31.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.289+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdfb8010430 con 0x7fdfc81041a0 2024-05-14T17:01:31.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.290+0000 7fdfbeffd700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdfb40b2910 0x7fdfb40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.290+0000 7fdfcdbd4700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdfb40b2910 0x7fdfb40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.290+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fdfb80e2b10 con 0x7fdfc81041a0 2024-05-14T17:01:31.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.290+0000 7fdfbeffd700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] conn(0x7fdfb40bc450 0x7fdfb40be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.290+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 --> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfb800e1a0 con 0x7fdfb40bc450 2024-05-14T17:01:31.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.290+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fdfb80e2f00 con 0x7fdfc81041a0 2024-05-14T17:01:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.291+0000 7fdfce3d5700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] conn(0x7fdfb40bc450 0x7fdfb40be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.291+0000 7fdfcdbd4700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdfb40b2910 0x7fdfb40b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000c00 tx=0x7fdfc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:31.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.291+0000 7fdfce3d5700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] conn(0x7fdfb40bc450 0x7fdfb40be8f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.292+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== osd.11 v2:172.21.15.193:6824/3625177067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfb800e1a0 con 0x7fdfb40bc450 2024-05-14T17:01:31.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.308+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 --> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdfc8064110 con 0x7fdfb40bc450 2024-05-14T17:01:31.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfbeffd700 1 -- 172.21.15.99:0/1266375011 <== osd.11 v2:172.21.15.193:6824/3625177067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdfc8064110 con 0x7fdfb40bc450 2024-05-14T17:01:31.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] conn(0x7fdfb40bc450 msgr2=0x7fdfb40be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] conn(0x7fdfb40bc450 0x7fdfb40be8f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdfb40b2910 msgr2=0x7fdfb40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fdfb40b2910 0x7fdfb40b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000c00 tx=0x7fdfc4005fb0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81041a0 msgr2=0x7fdfc819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 --2- 172.21.15.99:0/1266375011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfc81041a0 0x7fdfc819fd40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800e9a0 tx=0x7fdfb800ecb0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfce3d5700 1 -- 172.21.15.99:0/1266375011 reap_dead start 2024-05-14T17:01:31.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 shutdown_connections 2024-05-14T17:01:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 >> 172.21.15.99:0/1266375011 conn(0x7fdfc807a6a0 msgr2=0x7fdfc81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:31.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 shutdown_connections 2024-05-14T17:01:31.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:30.309+0000 7fdfcfe38700 1 -- 172.21.15.99:0/1266375011 wait complete. 2024-05-14T17:01:31.990 INFO:teuthology.orchestra.run.smithi099.stdout:309237645345 2024-05-14T17:01:32.001 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-14T17:01:32.286 INFO:teuthology.orchestra.run.smithi099.stdout:137438953537 2024-05-14T17:01:32.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-14T17:01:32.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:32 smithi193 ceph-mon[35962]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:32.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:32 smithi099 ceph-mon[31851]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:32.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:32 smithi150 ceph-mon[36204]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:32.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.235+0000 7fed34c9a700 1 -- 172.21.15.99:0/4030678889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102df0 msgr2=0x7fed3010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:32.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.235+0000 7fed34c9a700 1 --2- 172.21.15.99:0/4030678889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102df0 0x7fed3010f2e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fed2800b0a0 tx=0x7fed2800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 -- 172.21.15.99:0/4030678889 shutdown_connections 2024-05-14T17:01:32.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 --2- 172.21.15.99:0/4030678889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102df0 0x7fed3010f2e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 --2- 172.21.15.99:0/4030678889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108510 0x7fed301088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 --2- 172.21.15.99:0/4030678889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed30102430 0x7fed301028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 -- 172.21.15.99:0/4030678889 >> 172.21.15.99:0/4030678889 conn(0x7fed300793b0 msgr2=0x7fed300797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:33.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 -- 172.21.15.99:0/4030678889 shutdown_connections 2024-05-14T17:01:33.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 -- 172.21.15.99:0/4030678889 wait complete. 2024-05-14T17:01:33.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.236+0000 7fed34c9a700 1 Processor -- start 2024-05-14T17:01:33.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 -- start start 2024-05-14T17:01:33.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102430 0x7fed30104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed30102df0 0x7fed301045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108510 0x7fed30104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112710 con 0x7fed30102430 2024-05-14T17:01:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112590 con 0x7fed30108510 2024-05-14T17:01:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed34c9a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fed30112890 con 0x7fed30102df0 2024-05-14T17:01:33.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102430 0x7fed30104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102430 0x7fed30104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38614/0 (socket says 172.21.15.99:38614) 2024-05-14T17:01:33.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 -- 172.21.15.99:0/1117335111 learned_addr learned my addr 172.21.15.99:0/1117335111 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:33.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2ed9d700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108510 0x7fed30104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2dd9b700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed30102df0 0x7fed301045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 -- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed30102df0 msgr2=0x7fed301045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed30102df0 0x7fed301045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 -- 172.21.15.99:0/1117335111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108510 msgr2=0x7fed30104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108510 0x7fed30104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2e59c700 1 -- 172.21.15.99:0/1117335111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed301a8250 con 0x7fed30102430 2024-05-14T17:01:33.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.237+0000 7fed2ed9d700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed30108510 0x7fed30104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.238+0000 7fed2e59c700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102430 0x7fed30104070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fed2000b530 tx=0x7fed2000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:33.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.238+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20015020 con 0x7fed30102430 2024-05-14T17:01:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.238+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed301a8540 con 0x7fed30102430 2024-05-14T17:01:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.238+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed20011ce0 con 0x7fed30102430 2024-05-14T17:01:33.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.238+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20010d20 con 0x7fed30102430 2024-05-14T17:01:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.238+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301a8ad0 con 0x7fed30102430 2024-05-14T17:01:33.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.239+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed30063c50 con 0x7fed30102430 2024-05-14T17:01:33.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.241+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fed20018020 con 0x7fed30102430 2024-05-14T17:01:33.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed1f7fe700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fed180b2910 0x7fed180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed2dd9b700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fed180b2910 0x7fed180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fed200e2250 con 0x7fed30102430 2024-05-14T17:01:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed1f7fe700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] conn(0x7fed180bc450 0x7fed180be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 --> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed2000fec0 con 0x7fed180bc450 2024-05-14T17:01:33.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fed200e26a0 con 0x7fed30102430 2024-05-14T17:01:33.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.242+0000 7fed2dd9b700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fed180b2910 0x7fed180b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed24006fd0 tx=0x7fed24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:33.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.243+0000 7fed2ed9d700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] conn(0x7fed180bc450 0x7fed180be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.243+0000 7fed2ed9d700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] conn(0x7fed180bc450 0x7fed180be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:33.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.244+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== osd.2 v2:172.21.15.193:6800/2485896294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed2000fec0 con 0x7fed180bc450 2024-05-14T17:01:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.261+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 --> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed30064050 con 0x7fed180bc450 2024-05-14T17:01:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.261+0000 7fed1f7fe700 1 -- 172.21.15.99:0/1117335111 <== osd.2 v2:172.21.15.193:6800/2485896294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed30064050 con 0x7fed180bc450 2024-05-14T17:01:33.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.261+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] conn(0x7fed180bc450 msgr2=0x7fed180be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] conn(0x7fed180bc450 0x7fed180be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fed180b2910 msgr2=0x7fed180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fed180b2910 0x7fed180b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed24006fd0 tx=0x7fed24008040 comp rx=0 tx=0).stop 2024-05-14T17:01:33.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102430 msgr2=0x7fed30104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 --2- 172.21.15.99:0/1117335111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed30102430 0x7fed30104070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fed2000b530 tx=0x7fed2000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed2ed9d700 1 -- 172.21.15.99:0/1117335111 reap_dead start 2024-05-14T17:01:33.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 shutdown_connections 2024-05-14T17:01:33.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 >> 172.21.15.99:0/1117335111 conn(0x7fed300793b0 msgr2=0x7fed3006be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:33.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 shutdown_connections 2024-05-14T17:01:33.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:31.262+0000 7fed34c9a700 1 -- 172.21.15.99:0/1117335111 wait complete. 2024-05-14T17:01:33.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 -- 172.21.15.99:0/3917372142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102e30 msgr2=0x7f67c410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/3917372142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102e30 0x7f67c410f2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b0a0 tx=0x7f67bc00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 -- 172.21.15.99:0/3917372142 shutdown_connections 2024-05-14T17:01:33.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/3917372142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102e30 0x7f67c410f2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/3917372142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c41084b0 0x7f67c4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/3917372142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67c4102470 0x7f67c41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 -- 172.21.15.99:0/3917372142 >> 172.21.15.99:0/3917372142 conn(0x7f67c40793f0 msgr2=0x7f67c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 -- 172.21.15.99:0/3917372142 shutdown_connections 2024-05-14T17:01:33.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.332+0000 7f67cc2c0700 1 -- 172.21.15.99:0/3917372142 wait complete. 2024-05-14T17:01:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 Processor -- start 2024-05-14T17:01:33.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 -- start start 2024-05-14T17:01:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102470 0x7f67c419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4102e30 0x7f67c419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67c41084b0 0x7f67c41a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112660 con 0x7f67c4102470 2024-05-14T17:01:33.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41124e0 con 0x7f67c4102e30 2024-05-14T17:01:33.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67cc2c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67c41127e0 con 0x7f67c41084b0 2024-05-14T17:01:33.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67ca05c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102470 0x7f67c419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67ca05c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102470 0x7f67c419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38628/0 (socket says 172.21.15.99:38628) 2024-05-14T17:01:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.333+0000 7f67ca05c700 1 -- 172.21.15.99:0/2533338408 learned_addr learned my addr 172.21.15.99:0/2533338408 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67c985b700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4102e30 0x7f67c419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67ca05c700 1 -- 172.21.15.99:0/2533338408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67c41084b0 msgr2=0x7f67c41a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67ca05c700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67c41084b0 0x7f67c41a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67ca05c700 1 -- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4102e30 msgr2=0x7f67c419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67ca05c700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67c4102e30 0x7f67c419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67ca05c700 1 -- 172.21.15.99:0/2533338408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a6020 con 0x7f67c4102470 2024-05-14T17:01:34.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67ca05c700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102470 0x7f67c419aa60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f67c000b530 tx=0x7f67c000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:34.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0015020 con 0x7f67c4102470 2024-05-14T17:01:34.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c0011ce0 con 0x7f67c4102470 2024-05-14T17:01:34.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a6250 con 0x7f67c4102470 2024-05-14T17:01:34.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0010d20 con 0x7f67c4102470 2024-05-14T17:01:34.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.334+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a67e0 con 0x7f67c4102470 2024-05-14T17:01:34.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.335+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67c4063c50 con 0x7f67c4102470 2024-05-14T17:01:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.336+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f67c0018020 con 0x7f67c4102470 2024-05-14T17:01:34.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.337+0000 7f67bb7fe700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f67b00b2990 0x7f67b00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:34.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.337+0000 7f67c985b700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f67b00b2990 0x7f67b00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:34.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.337+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f67c00e2180 con 0x7f67c4102470 2024-05-14T17:01:34.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.337+0000 7f67bb7fe700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] conn(0x7f67b00bc4d0 0x7f67b00be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:34.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.337+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 --> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67c001be50 con 0x7f67b00bc4d0 2024-05-14T17:01:34.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.337+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f67c00e25d0 con 0x7f67c4102470 2024-05-14T17:01:34.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.338+0000 7f67c985b700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f67b00b2990 0x7f67b00b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67c4079a60 tx=0x7f67b4005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:34.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.338+0000 7f67ca85d700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] conn(0x7f67b00bc4d0 0x7f67b00be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:34.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.338+0000 7f67ca85d700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] conn(0x7f67b00bc4d0 0x7f67b00be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:34.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.339+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== osd.7 v2:172.21.15.150:6816/1186953586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67c001be50 con 0x7f67b00bc4d0 2024-05-14T17:01:34.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.355+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 --> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67c4064050 con 0x7f67b00bc4d0 2024-05-14T17:01:34.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67bb7fe700 1 -- 172.21.15.99:0/2533338408 <== osd.7 v2:172.21.15.150:6816/1186953586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f67c4064050 con 0x7f67b00bc4d0 2024-05-14T17:01:34.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] conn(0x7f67b00bc4d0 msgr2=0x7f67b00be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] conn(0x7f67b00bc4d0 0x7f67b00be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f67b00b2990 msgr2=0x7f67b00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f67b00b2990 0x7f67b00b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67c4079a60 tx=0x7f67b4005eb0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102470 msgr2=0x7f67c419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67cc2c0700 1 --2- 172.21.15.99:0/2533338408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67c4102470 0x7f67c419aa60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f67c000b530 tx=0x7f67c000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.356+0000 7f67ca85d700 1 -- 172.21.15.99:0/2533338408 reap_dead start 2024-05-14T17:01:34.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.357+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 shutdown_connections 2024-05-14T17:01:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.357+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 >> 172.21.15.99:0/2533338408 conn(0x7f67c40793f0 msgr2=0x7f67c4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:34.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.357+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 shutdown_connections 2024-05-14T17:01:34.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:33.357+0000 7f67cc2c0700 1 -- 172.21.15.99:0/2533338408 wait complete. 2024-05-14T17:01:34.423 INFO:teuthology.orchestra.run.smithi099.stdout:77309411405 2024-05-14T17:01:34.435 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-14T17:01:34.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:34 smithi099 ceph-mon[31851]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:34.532 INFO:teuthology.orchestra.run.smithi099.stdout:34359738454 2024-05-14T17:01:34.544 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-14T17:01:34.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:34 smithi193 ceph-mon[35962]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:34.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:34 smithi150 ceph-mon[36204]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:34.918 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:34.958 INFO:teuthology.orchestra.run.smithi099.stdout:55834574929 2024-05-14T17:01:34.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-14T17:01:35.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/2428459881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006c470 msgr2=0x7f4cf01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/2428459881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006c470 0x7f4cf01103c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800b0a0 tx=0x7f4ce800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/2428459881 shutdown_connections 2024-05-14T17:01:35.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/2428459881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006c470 0x7f4cf01103c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/2428459881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cf01095a0 0x7f4cf0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/2428459881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf006bab0 0x7f4cf006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/2428459881 >> 172.21.15.99:0/2428459881 conn(0x7f4cf00793b0 msgr2=0x7f4cf00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:35.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/2428459881 shutdown_connections 2024-05-14T17:01:35.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.153+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/2428459881 wait complete. 2024-05-14T17:01:35.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.154+0000 7f4cf4d0d700 1 Processor -- start 2024-05-14T17:01:35.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.154+0000 7f4cf4d0d700 1 -- start start 2024-05-14T17:01:35.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.154+0000 7f4cf4d0d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006bab0 0x7f4cf019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cf006c470 0x7f4cf019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf01095a0 0x7f4cf0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf01136d0 con 0x7f4cf006bab0 2024-05-14T17:01:35.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0113550 con 0x7f4cf006c470 2024-05-14T17:01:35.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0113850 con 0x7f4cf01095a0 2024-05-14T17:01:35.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006bab0 0x7f4cf019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006bab0 0x7f4cf019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32804/0 (socket says 172.21.15.99:32804) 2024-05-14T17:01:35.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 -- 172.21.15.99:0/568361243 learned_addr learned my addr 172.21.15.99:0/568361243 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:35.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4ceed9d700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf01095a0 0x7f4cf0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cedd9b700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cf006c470 0x7f4cf019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 -- 172.21.15.99:0/568361243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf01095a0 msgr2=0x7f4cf0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4cf01095a0 0x7f4cf0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 -- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cf006c470 msgr2=0x7f4cf019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cf006c470 0x7f4cf019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 -- 172.21.15.99:0/568361243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf019a190 con 0x7f4cf006bab0 2024-05-14T17:01:35.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cee59c700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006bab0 0x7f4cf019f820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000d950 tx=0x7f4ce000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0016070 con 0x7f4cf006bab0 2024-05-14T17:01:35.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf019a480 con 0x7f4cf006bab0 2024-05-14T17:01:35.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce000fd40 con 0x7f4cf006bab0 2024-05-14T17:01:35.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000e3b0 con 0x7f4cf006bab0 2024-05-14T17:01:35.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.155+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf01a66f0 con 0x7f4cf006bab0 2024-05-14T17:01:35.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.156+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cf0063cd0 con 0x7f4cf006bab0 2024-05-14T17:01:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.158+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4ce0019070 con 0x7f4cf006bab0 2024-05-14T17:01:35.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cdf7fe700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4cd80b2910 0x7f4cd80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cedd9b700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4cd80b2910 0x7f4cd80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f4ce00e2380 con 0x7f4cf006bab0 2024-05-14T17:01:35.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cdf7fe700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] conn(0x7f4cd80bc450 0x7f4cd80be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 --> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cd80befc0 con 0x7f4cd80bc450 2024-05-14T17:01:35.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4ce00e27d0 con 0x7f4cf006bab0 2024-05-14T17:01:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4ceed9d700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] conn(0x7f4cd80bc450 0x7f4cd80be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.159+0000 7f4cedd9b700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4cd80b2910 0x7f4cd80b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0079a20 tx=0x7f4ce400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.160+0000 7f4ceed9d700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] conn(0x7f4cd80bc450 0x7f4cd80be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.161+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== osd.10 v2:172.21.15.150:6824/2780871308 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cd80befc0 con 0x7f4cd80bc450 2024-05-14T17:01:35.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.178+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 --> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4cf00640d0 con 0x7f4cd80bc450 2024-05-14T17:01:35.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.178+0000 7f4cdf7fe700 1 -- 172.21.15.99:0/568361243 <== osd.10 v2:172.21.15.150:6824/2780871308 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4cf00640d0 con 0x7f4cd80bc450 2024-05-14T17:01:35.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] conn(0x7f4cd80bc450 msgr2=0x7f4cd80be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] conn(0x7f4cd80bc450 0x7f4cd80be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4cd80b2910 msgr2=0x7f4cd80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4cd80b2910 0x7f4cd80b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0079a20 tx=0x7f4ce400b3c0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006bab0 msgr2=0x7f4cf019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 --2- 172.21.15.99:0/568361243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf006bab0 0x7f4cf019f820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000d950 tx=0x7f4ce000dc60 comp rx=0 tx=0).stop 2024-05-14T17:01:35.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4ceed9d700 1 -- 172.21.15.99:0/568361243 reap_dead start 2024-05-14T17:01:35.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 shutdown_connections 2024-05-14T17:01:35.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 >> 172.21.15.99:0/568361243 conn(0x7f4cf00793b0 msgr2=0x7f4cf01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:35.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 shutdown_connections 2024-05-14T17:01:35.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:35.179+0000 7f4cf4d0d700 1 -- 172.21.15.99:0/568361243 wait complete. 2024-05-14T17:01:35.988 INFO:teuthology.orchestra.run.smithi099.stdout:210453397556 2024-05-14T17:01:36.001 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-14T17:01:36.235 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:36.315 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:36.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:36 smithi193 ceph-mon[35962]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:36.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:36 smithi099 ceph-mon[31851]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:36.705 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:36 smithi150 ceph-mon[36204]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:36.723 INFO:teuthology.orchestra.run.smithi099.stdout:283467841574 2024-05-14T17:01:36.771 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-14T17:01:37.284 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:37.339 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:38.619 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:38 smithi099 ceph-mon[31851]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:01:38.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:38 smithi193 ceph-mon[35962]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:01:38.801 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:38 smithi150 ceph-mon[36204]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-14T17:01:40.390 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:40 smithi099 ceph-mon[31851]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s 2024-05-14T17:01:40.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:40 smithi150 ceph-mon[36204]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s 2024-05-14T17:01:40.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:40 smithi193 ceph-mon[35962]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s 2024-05-14T17:01:41.414 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:41.535 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:42.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:42 smithi193 ceph-mon[35962]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:01:42.636 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:42 smithi099 ceph-mon[31851]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:01:42.645 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:42.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:42 smithi150 ceph-mon[36204]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:01:43.040 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:43.074 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:43.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 -- 172.21.15.99:0/402612699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103600 msgr2=0x7f4f54103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:43.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/402612699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103600 0x7f4f54103a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4f400099c0 tx=0x7f4f40009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 -- 172.21.15.99:0/402612699 shutdown_connections 2024-05-14T17:01:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/402612699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103fc0 0x7f4f541103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/402612699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f541095e0 0x7f4f541099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/402612699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103600 0x7f4f54103a80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 -- 172.21.15.99:0/402612699 >> 172.21.15.99:0/402612699 conn(0x7f4f5407a700 msgr2=0x7f4f5407ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:43.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 -- 172.21.15.99:0/402612699 shutdown_connections 2024-05-14T17:01:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.749+0000 7f4f5ab75700 1 -- 172.21.15.99:0/402612699 wait complete. 2024-05-14T17:01:43.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 Processor -- start 2024-05-14T17:01:43.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 -- start start 2024-05-14T17:01:43.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103600 0x7f4f5419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 0x7f4f5419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:43.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f541095e0 0x7f4f54199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:43.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541137c0 con 0x7f4f54103fc0 2024-05-14T17:01:43.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54113640 con 0x7f4f541095e0 2024-05-14T17:01:43.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.750+0000 7f4f5ab75700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54113940 con 0x7f4f54103600 2024-05-14T17:01:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 0x7f4f5419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:43.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 0x7f4f5419ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32820/0 (socket says 172.21.15.99:32820) 2024-05-14T17:01:43.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 -- 172.21.15.99:0/1891698946 learned_addr learned my addr 172.21.15.99:0/1891698946 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:43.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f58911700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103600 0x7f4f5419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f59112700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f541095e0 0x7f4f54199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 -- 172.21.15.99:0/1891698946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103600 msgr2=0x7f4f5419fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103600 0x7f4f5419fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 -- 172.21.15.99:0/1891698946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f541095e0 msgr2=0x7f4f54199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f541095e0 0x7f4f54199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 -- 172.21.15.99:0/1891698946 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5419a400 con 0x7f4f54103fc0 2024-05-14T17:01:44.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f58911700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103600 0x7f4f5419fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:44.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f53fff700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 0x7f4f5419ff90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000b580 tx=0x7f4f400037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:44.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40027070 con 0x7f4f54103fc0 2024-05-14T17:01:44.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f40004110 con 0x7f4f54103fc0 2024-05-14T17:01:44.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5419a690 con 0x7f4f54103fc0 2024-05-14T17:01:44.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.751+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000f460 con 0x7f4f54103fc0 2024-05-14T17:01:44.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.752+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a69d0 con 0x7f4f54103fc0 2024-05-14T17:01:44.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.752+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5410ba70 con 0x7f4f54103fc0 2024-05-14T17:01:44.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.754+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4f4002a070 con 0x7f4f54103fc0 2024-05-14T17:01:44.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.755+0000 7f4f51ffb700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4f440b2990 0x7f4f440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.755+0000 7f4f58911700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4f440b2990 0x7f4f440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.755+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f4f400e3770 con 0x7f4f54103fc0 2024-05-14T17:01:44.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.755+0000 7f4f58911700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4f440b2990 0x7f4f440b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4f480099d0 tx=0x7f4f48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:44.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.763+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f40031070 con 0x7f4f54103fc0 2024-05-14T17:01:44.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.962+0000 7fef4f839700 1 -- 172.21.15.99:0/3514320547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806bab0 msgr2=0x7fef4806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.962+0000 7fef4f839700 1 --2- 172.21.15.99:0/3514320547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806bab0 0x7fef4806bf30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef38009990 tx=0x7fef38009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 -- 172.21.15.99:0/3514320547 shutdown_connections 2024-05-14T17:01:44.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 --2- 172.21.15.99:0/3514320547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef4806c470 0x7fef481103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 --2- 172.21.15.99:0/3514320547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef481095a0 0x7fef48109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 --2- 172.21.15.99:0/3514320547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806bab0 0x7fef4806bf30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 -- 172.21.15.99:0/3514320547 >> 172.21.15.99:0/3514320547 conn(0x7fef480793b0 msgr2=0x7fef480797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:44.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 -- 172.21.15.99:0/3514320547 shutdown_connections 2024-05-14T17:01:44.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 -- 172.21.15.99:0/3514320547 wait complete. 2024-05-14T17:01:44.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 Processor -- start 2024-05-14T17:01:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 -- start start 2024-05-14T17:01:44.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.963+0000 7fef4f839700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef4806bab0 0x7fef48102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4f839700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 0x7fef48102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4f839700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef481095a0 0x7fef48102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4f839700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef48113750 con 0x7fef4806c470 2024-05-14T17:01:44.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4f839700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef481135d0 con 0x7fef481095a0 2024-05-14T17:01:44.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4f839700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef481138d0 con 0x7fef4806bab0 2024-05-14T17:01:44.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 0x7fef48102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 0x7fef48102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32830/0 (socket says 172.21.15.99:32830) 2024-05-14T17:01:44.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 -- 172.21.15.99:0/1506137171 learned_addr learned my addr 172.21.15.99:0/1506137171 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:44.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4d5d5700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef4806bab0 0x7fef48102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4ddd6700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef481095a0 0x7fef48102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 -- 172.21.15.99:0/1506137171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef4806bab0 msgr2=0x7fef48102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef4806bab0 0x7fef48102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 -- 172.21.15.99:0/1506137171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef481095a0 msgr2=0x7fef48102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef481095a0 0x7fef48102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 -- 172.21.15.99:0/1506137171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef481a81b0 con 0x7fef4806c470 2024-05-14T17:01:44.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4ddd6700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef481095a0 0x7fef48102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:44.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4cdd4700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 0x7fef48102a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fef38000c00 tx=0x7fef380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:44.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38027070 con 0x7fef4806c470 2024-05-14T17:01:44.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef481a8440 con 0x7fef4806c470 2024-05-14T17:01:44.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef38003f20 con 0x7fef4806c470 2024-05-14T17:01:44.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.964+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38022cf0 con 0x7fef4806c470 2024-05-14T17:01:44.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.965+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a88f0 con 0x7fef4806c470 2024-05-14T17:01:44.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.966+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4810ba80 con 0x7fef4806c470 2024-05-14T17:01:44.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.967+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fef3802a070 con 0x7fef4806c470 2024-05-14T17:01:44.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.968+0000 7fef3e7fc700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef340b2910 0x7fef340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.968+0000 7fef4d5d5700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef340b2910 0x7fef340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.968+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fef380e3470 con 0x7fef4806c470 2024-05-14T17:01:44.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.968+0000 7fef4d5d5700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef340b2910 0x7fef340b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef44004630 tx=0x7fef4400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:44.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.976+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef38013070 con 0x7fef4806c470 2024-05-14T17:01:44.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.998+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4f5419afe0 con 0x7f4f54103fc0 2024-05-14T17:01:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:43.998+0000 7f4f51ffb700 1 -- 172.21.15.99:0/1891698946 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4f40013070 con 0x7f4f54103fc0 2024-05-14T17:01:44.781 INFO:teuthology.orchestra.run.smithi099.stdout:137438953540 2024-05-14T17:01:44.788 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:44 smithi150 ceph-mon[36204]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:44.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:44 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1891698946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T17:01:44.830 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:44 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1506137171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T17:01:44.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4f440b2990 msgr2=0x7f4f440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4f440b2990 0x7f4f440b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4f480099d0 tx=0x7f4f48008040 comp rx=0 tx=0).stop 2024-05-14T17:01:44.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 msgr2=0x7f4f5419ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 0x7f4f5419ff90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000b580 tx=0x7f4f400037c0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 shutdown_connections 2024-05-14T17:01:44.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4f440b2990 0x7f4f440b4e50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f541095e0 0x7f4f54199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f54103600 0x7f4f5419fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 --2- 172.21.15.99:0/1891698946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54103fc0 0x7f4f5419ff90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 >> 172.21.15.99:0/1891698946 conn(0x7f4f5407a700 msgr2=0x7f4f541022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:44.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 shutdown_connections 2024-05-14T17:01:44.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.001+0000 7f4f5ab75700 1 -- 172.21.15.99:0/1891698946 wait complete. 2024-05-14T17:01:44.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.195+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fef48103a30 con 0x7fef4806c470 2024-05-14T17:01:44.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.196+0000 7fef3e7fc700 1 -- 172.21.15.99:0/1506137171 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fef380ab0d0 con 0x7fef4806c470 2024-05-14T17:01:44.972 INFO:teuthology.orchestra.run.smithi099.stdout:309237645347 2024-05-14T17:01:44.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef340b2910 msgr2=0x7fef340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef340b2910 0x7fef340b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef44004630 tx=0x7fef4400b3c0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 msgr2=0x7fef48102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 0x7fef48102a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fef38000c00 tx=0x7fef380047b0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 shutdown_connections 2024-05-14T17:01:45.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fef340b2910 0x7fef340b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.198+0000 7fef4f839700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef481095a0 0x7fef48102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.199+0000 7fef4f839700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef4806bab0 0x7fef48102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.199+0000 7fef4f839700 1 --2- 172.21.15.99:0/1506137171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef4806c470 0x7fef48102a40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.199+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 >> 172.21.15.99:0/1506137171 conn(0x7fef480793b0 msgr2=0x7fef48101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:45.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.200+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 shutdown_connections 2024-05-14T17:01:45.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.200+0000 7fef4f839700 1 -- 172.21.15.99:0/1506137171 wait complete. 2024-05-14T17:01:45.132 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:44 smithi099 ceph-mon[31851]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:45.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:44 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1891698946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T17:01:45.150 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:44 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1506137171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T17:01:45.160 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:44 smithi193 ceph-mon[35962]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:45.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:44 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1891698946' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T17:01:45.212 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:44 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1506137171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T17:01:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1540488027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954109610 msgr2=0x7ff9541099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1540488027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954109610 0x7ff9541099f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9440099c0 tx=0x7ff944009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1540488027 shutdown_connections 2024-05-14T17:01:45.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1540488027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954102450 0x7ff9541103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1540488027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954109610 0x7ff9541099f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1540488027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954101a90 0x7ff954101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1540488027 >> 172.21.15.99:0/1540488027 conn(0x7ff954078a40 msgr2=0x7ff9540ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:45.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1540488027 shutdown_connections 2024-05-14T17:01:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.869+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1540488027 wait complete. 2024-05-14T17:01:45.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 Processor -- start 2024-05-14T17:01:45.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 -- start start 2024-05-14T17:01:45.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954101a90 0x7ff954102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 0x7ff9541031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954109610 0x7ff9541036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff954113730 con 0x7ff954102450 2024-05-14T17:01:45.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541135b0 con 0x7ff954101a90 2024-05-14T17:01:45.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95c4f6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541138b0 con 0x7ff954109610 2024-05-14T17:01:45.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 0x7ff9541031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:45.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 0x7ff9541031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42824/0 (socket says 172.21.15.99:42824) 2024-05-14T17:01:45.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 -- 172.21.15.99:0/1481095399 learned_addr learned my addr 172.21.15.99:0/1481095399 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:45.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95a292700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954101a90 0x7ff954102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:45.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff95aa93700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954109610 0x7ff9541036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:45.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 -- 172.21.15.99:0/1481095399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954109610 msgr2=0x7ff9541036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954109610 0x7ff9541036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 -- 172.21.15.99:0/1481095399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954101a90 msgr2=0x7ff954102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.870+0000 7ff959a91700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954101a90 0x7ff954102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff959a91700 1 -- 172.21.15.99:0/1481095399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9541a8150 con 0x7ff954102450 2024-05-14T17:01:45.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff95a292700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954101a90 0x7ff954102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:45.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff959a91700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 0x7ff9541031b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff95000d980 tx=0x7ff95000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:45.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950017070 con 0x7ff954102450 2024-05-14T17:01:45.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff950004830 con 0x7ff954102450 2024-05-14T17:01:45.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9541a8440 con 0x7ff954102450 2024-05-14T17:01:45.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950005440 con 0x7ff954102450 2024-05-14T17:01:45.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.871+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a8930 con 0x7ff954102450 2024-05-14T17:01:45.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.872+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95406a230 con 0x7ff954102450 2024-05-14T17:01:45.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.873+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff9500055e0 con 0x7ff954102450 2024-05-14T17:01:45.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.874+0000 7ff94b7fe700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff9400b2910 0x7ff9400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.875+0000 7ff95a292700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff9400b2910 0x7ff9400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:45.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.875+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7ff9500e2710 con 0x7ff954102450 2024-05-14T17:01:45.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.875+0000 7ff95a292700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff9400b2910 0x7ff9400b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff944003800 tx=0x7ff944009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:45.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:44.878+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff950020020 con 0x7ff954102450 2024-05-14T17:01:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.101+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff95402e560 con 0x7ff954102450 2024-05-14T17:01:45.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.101+0000 7ff94b7fe700 1 -- 172.21.15.99:0/1481095399 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff9500aa2e0 con 0x7ff954102450 2024-05-14T17:01:45.797 INFO:teuthology.orchestra.run.smithi099.stdout:111669149770 2024-05-14T17:01:45.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:45 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1481095399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T17:01:45.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:45 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1481095399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T17:01:45.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.103+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff9400b2910 msgr2=0x7ff9400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.103+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff9400b2910 0x7ff9400b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff944003800 tx=0x7ff944009f10 comp rx=0 tx=0).stop 2024-05-14T17:01:45.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 msgr2=0x7ff9541031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 0x7ff9541031b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff95000d980 tx=0x7ff95000dd40 comp rx=0 tx=0).stop 2024-05-14T17:01:45.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 shutdown_connections 2024-05-14T17:01:45.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff9400b2910 0x7ff9400b4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff954109610 0x7ff9541036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff954101a90 0x7ff954102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 --2- 172.21.15.99:0/1481095399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff954102450 0x7ff9541031b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 >> 172.21.15.99:0/1481095399 conn(0x7ff954078a40 msgr2=0x7ff9541001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:45.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 shutdown_connections 2024-05-14T17:01:45.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:45.104+0000 7ff95c4f6700 1 -- 172.21.15.99:0/1481095399 wait complete. 2024-05-14T17:01:45.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:45 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1481095399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T17:01:46.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:46 smithi099 ceph-mon[31851]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:46.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:46 smithi150 ceph-mon[36204]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:46.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:46 smithi193 ceph-mon[35962]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:47.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645345 got 309237645347 for osd.11 2024-05-14T17:01:47.876 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:47.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953537 got 137438953540 for osd.4 2024-05-14T17:01:47.909 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:48.391 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:48 smithi099 ceph-mon[31851]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:48.618 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149765 got 111669149770 for osd.3 2024-05-14T17:01:48.628 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:48.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:48 smithi150 ceph-mon[36204]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:48.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:48 smithi193 ceph-mon[35962]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:48.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 -- 172.21.15.99:0/3857457074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c101e60 msgr2=0x7fcd3c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:48.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 --2- 172.21.15.99:0/3857457074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c101e60 0x7fcd3c102240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0099c0 tx=0x7fcd2c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 -- 172.21.15.99:0/3857457074 shutdown_connections 2024-05-14T17:01:48.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 --2- 172.21.15.99:0/3857457074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 0x7fcd3c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 --2- 172.21.15.99:0/3857457074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c101e60 0x7fcd3c102240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 --2- 172.21.15.99:0/3857457074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd3c102810 0x7fcd3c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 -- 172.21.15.99:0/3857457074 >> 172.21.15.99:0/3857457074 conn(0x7fcd3c078ab0 msgr2=0x7fcd3c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:48.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 -- 172.21.15.99:0/3857457074 shutdown_connections 2024-05-14T17:01:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.853+0000 7fcd412f9700 1 -- 172.21.15.99:0/3857457074 wait complete. 2024-05-14T17:01:48.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 Processor -- start 2024-05-14T17:01:48.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 -- start start 2024-05-14T17:01:48.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd3c101e60 0x7fcd3c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 0x7fcd3c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 0x7fcd3c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111da0 con 0x7fcd3c102810 2024-05-14T17:01:49.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111c20 con 0x7fcd3c101e60 2024-05-14T17:01:49.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd412f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c111f20 con 0x7fcd3c106ef0 2024-05-14T17:01:49.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd3a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 0x7fcd3c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd3a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 0x7fcd3c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42836/0 (socket says 172.21.15.99:42836) 2024-05-14T17:01:49.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd3a7fc700 1 -- 172.21.15.99:0/4217749971 learned_addr learned my addr 172.21.15.99:0/4217749971 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:49.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.854+0000 7fcd3affd700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd3c101e60 0x7fcd3c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3b7fe700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 0x7fcd3c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3a7fc700 1 -- 172.21.15.99:0/4217749971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 msgr2=0x7fcd3c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3a7fc700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 0x7fcd3c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3a7fc700 1 -- 172.21.15.99:0/4217749971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd3c101e60 msgr2=0x7fcd3c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3a7fc700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd3c101e60 0x7fcd3c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3a7fc700 1 -- 172.21.15.99:0/4217749971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c19fdc0 con 0x7fcd3c102810 2024-05-14T17:01:49.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3b7fe700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 0x7fcd3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:49.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd3a7fc700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 0x7fcd3c19aca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c9e0 tx=0x7fcd3000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:49.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30007aa0 con 0x7fcd3c102810 2024-05-14T17:01:49.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd30007c40 con 0x7fcd3c102810 2024-05-14T17:01:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30018660 con 0x7fcd3c102810 2024-05-14T17:01:49.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a00b0 con 0x7fcd3c102810 2024-05-14T17:01:49.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.855+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a6770 con 0x7fcd3c102810 2024-05-14T17:01:49.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.856+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c1032f0 con 0x7fcd3c102810 2024-05-14T17:01:49.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.858+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcd30018800 con 0x7fcd3c102810 2024-05-14T17:01:49.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.859+0000 7fcd23fff700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd240b2910 0x7fcd240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.859+0000 7fcd3affd700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd240b2910 0x7fcd240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.859+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fcd300e2600 con 0x7fcd3c102810 2024-05-14T17:01:49.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.859+0000 7fcd3affd700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd240b2910 0x7fcd240b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009990 tx=0x7fcd2c009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:49.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:48.862+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd30014030 con 0x7fcd3c102810 2024-05-14T17:01:49.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.075+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fcd3c1a6970 con 0x7fcd3c102810 2024-05-14T17:01:49.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.076+0000 7fcd23fff700 1 -- 172.21.15.99:0/4217749971 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd300aa260 con 0x7fcd3c102810 2024-05-14T17:01:49.385 INFO:teuthology.orchestra.run.smithi099.stdout:261993005101 2024-05-14T17:01:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd240b2910 msgr2=0x7fcd240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd240b2910 0x7fcd240b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009990 tx=0x7fcd2c009650 comp rx=0 tx=0).stop 2024-05-14T17:01:49.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 msgr2=0x7fcd3c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 0x7fcd3c19aca0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c9e0 tx=0x7fcd3000ccf0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 shutdown_connections 2024-05-14T17:01:49.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcd240b2910 0x7fcd240b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd3c106ef0 0x7fcd3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd3c101e60 0x7fcd3c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 --2- 172.21.15.99:0/4217749971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3c102810 0x7fcd3c19aca0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 >> 172.21.15.99:0/4217749971 conn(0x7fcd3c078ab0 msgr2=0x7fcd3c106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 shutdown_connections 2024-05-14T17:01:49.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.078+0000 7fcd412f9700 1 -- 172.21.15.99:0/4217749971 wait complete. 2024-05-14T17:01:49.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:49 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4217749971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T17:01:49.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:49 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4217749971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T17:01:49.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:49 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4217749971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T17:01:49.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/1532749442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b41095b0 msgr2=0x7fe0b4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/1532749442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b41095b0 0x7fe0b4109990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009990 tx=0x7fe0a4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/1532749442 shutdown_connections 2024-05-14T17:01:50.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/1532749442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b406c650 0x7fe0b41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/1532749442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b41095b0 0x7fe0b4109990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/1532749442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b406bc90 0x7fe0b406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/1532749442 >> 172.21.15.99:0/1532749442 conn(0x7fe0b4079410 msgr2=0x7fe0b4079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/1532749442 shutdown_connections 2024-05-14T17:01:50.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/1532749442 wait complete. 2024-05-14T17:01:50.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 Processor -- start 2024-05-14T17:01:50.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.952+0000 7fe0bb7a9700 1 -- start start 2024-05-14T17:01:50.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 0x7fe0b419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b406c650 0x7fe0b419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b41095b0 0x7fe0b41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b41136e0 con 0x7fe0b406bc90 2024-05-14T17:01:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4113560 con 0x7fe0b41095b0 2024-05-14T17:01:50.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4113860 con 0x7fe0b406c650 2024-05-14T17:01:50.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 0x7fe0b419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 0x7fe0b419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42862/0 (socket says 172.21.15.99:42862) 2024-05-14T17:01:50.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 -- 172.21.15.99:0/833637879 learned_addr learned my addr 172.21.15.99:0/833637879 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b8d44700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b406c650 0x7fe0b419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9d46700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b41095b0 0x7fe0b41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 -- 172.21.15.99:0/833637879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b406c650 msgr2=0x7fe0b419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:50.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b406c650 0x7fe0b419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 -- 172.21.15.99:0/833637879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b41095b0 msgr2=0x7fe0b41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:50.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b41095b0 0x7fe0b41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 -- 172.21.15.99:0/833637879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b419a120 con 0x7fe0b406bc90 2024-05-14T17:01:50.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9d46700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b41095b0 0x7fe0b41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0b9545700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 0x7fe0b419f800 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4000c00 tx=0x7fe0a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:50.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4027070 con 0x7fe0b406bc90 2024-05-14T17:01:50.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a4003f20 con 0x7fe0b406bc90 2024-05-14T17:01:50.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b419a3b0 con 0x7fe0b406bc90 2024-05-14T17:01:50.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.953+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4022cf0 con 0x7fe0b406bc90 2024-05-14T17:01:50.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.954+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b41a66d0 con 0x7fe0b406bc90 2024-05-14T17:01:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.954+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b410ba90 con 0x7fe0b406bc90 2024-05-14T17:01:50.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.956+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe0a402a070 con 0x7fe0b406bc90 2024-05-14T17:01:50.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.956+0000 7fe0aa7fc700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe0a00b2910 0x7fe0a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.956+0000 7fe0b8d44700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe0a00b2910 0x7fe0a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.956+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fe0a40e3170 con 0x7fe0b406bc90 2024-05-14T17:01:50.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.957+0000 7fe0b8d44700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe0a00b2910 0x7fe0a00b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4079a80 tx=0x7fe0b000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:50.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:49.961+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0a4013070 con 0x7fe0b406bc90 2024-05-14T17:01:50.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.176+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe0b410b470 con 0x7fe0b406bc90 2024-05-14T17:01:50.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.176+0000 7fe0aa7fc700 1 -- 172.21.15.99:0/833637879 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0a40aadd0 con 0x7fe0b406bc90 2024-05-14T17:01:50.524 INFO:teuthology.orchestra.run.smithi099.stdout:188978561085 2024-05-14T17:01:50.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe0a00b2910 msgr2=0x7fe0a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:50.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe0a00b2910 0x7fe0a00b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4079a80 tx=0x7fe0b000a380 comp rx=0 tx=0).stop 2024-05-14T17:01:50.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 msgr2=0x7fe0b419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:50.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 0x7fe0b419f800 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4000c00 tx=0x7fe0a40047b0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 shutdown_connections 2024-05-14T17:01:50.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fe0a00b2910 0x7fe0a00b4dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b41095b0 0x7fe0b41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b406bc90 0x7fe0b419f800 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 --2- 172.21.15.99:0/833637879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b406c650 0x7fe0b419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 >> 172.21.15.99:0/833637879 conn(0x7fe0b4079410 msgr2=0x7fe0b4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 shutdown_connections 2024-05-14T17:01:50.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:50.178+0000 7fe0bb7a9700 1 -- 172.21.15.99:0/833637879 wait complete. 2024-05-14T17:01:50.706 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:50 smithi099 ceph-mon[31851]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-14T17:01:50.715 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:50 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/833637879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T17:01:50.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:50 smithi150 ceph-mon[36204]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-14T17:01:50.734 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:50 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/833637879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T17:01:50.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:50 smithi193 ceph-mon[35962]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-14T17:01:50.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:50 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/833637879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T17:01:51.573 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005097 got 261993005101 for osd.9 2024-05-14T17:01:51.591 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:52.679 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:52 smithi099 ceph-mon[31851]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T17:01:52.709 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:52 smithi150 ceph-mon[36204]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T17:01:52.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:52 smithi193 ceph-mon[35962]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T17:01:53.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561080 got 188978561085 for osd.6 2024-05-14T17:01:53.632 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:54.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 -- 172.21.15.99:0/4010152549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 msgr2=0x7f1540110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:54.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 --2- 172.21.15.99:0/4010152549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f1540110470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f153800b0d0 tx=0x7f153800b3e0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 -- 172.21.15.99:0/4010152549 shutdown_connections 2024-05-14T17:01:54.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 --2- 172.21.15.99:0/4010152549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f1540110470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 --2- 172.21.15.99:0/4010152549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 0x7f1540103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 --2- 172.21.15.99:0/4010152549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15401041a0 0x7f15401081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 -- 172.21.15.99:0/4010152549 >> 172.21.15.99:0/4010152549 conn(0x7f154007a6a0 msgr2=0x7f154007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:54.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 -- 172.21.15.99:0/4010152549 shutdown_connections 2024-05-14T17:01:54.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.263+0000 7f1547be0700 1 -- 172.21.15.99:0/4010152549 wait complete. 2024-05-14T17:01:54.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 Processor -- start 2024-05-14T17:01:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 -- start start 2024-05-14T17:01:54.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 0x7f154019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:54.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15401041a0 0x7f154019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:54.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f15401998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:54.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15401136c0 con 0x7f15401088f0 2024-05-14T17:01:54.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1540113540 con 0x7f15401037f0 2024-05-14T17:01:54.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f1547be0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1540113840 con 0x7f15401041a0 2024-05-14T17:01:54.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f15401998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f15401998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42878/0 (socket says 172.21.15.99:42878) 2024-05-14T17:01:54.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 -- 172.21.15.99:0/3803440310 learned_addr learned my addr 172.21.15.99:0/3803440310 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:54.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154517b700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15401041a0 0x7f154019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154597c700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 0x7f154019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 -- 172.21.15.99:0/3803440310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15401041a0 msgr2=0x7f154019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:54.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15401041a0 0x7f154019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 -- 172.21.15.99:0/3803440310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 msgr2=0x7f154019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:54.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 0x7f154019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154617d700 1 -- 172.21.15.99:0/3803440310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154019a020 con 0x7f15401088f0 2024-05-14T17:01:54.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.264+0000 7f154597c700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 0x7f154019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:54.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.265+0000 7f154617d700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f15401998c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15380095c0 tx=0x7f1538007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:54.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.265+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153801f070 con 0x7f15401088f0 2024-05-14T17:01:54.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.265+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1538003c00 con 0x7f15401088f0 2024-05-14T17:01:54.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.265+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154019a2b0 con 0x7f15401088f0 2024-05-14T17:01:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.265+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153801b6a0 con 0x7f15401088f0 2024-05-14T17:01:54.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.265+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15401a65d0 con 0x7f15401088f0 2024-05-14T17:01:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.266+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154006a230 con 0x7f15401088f0 2024-05-14T17:01:54.716 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:54 smithi150 ceph-mon[36204]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:54.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.267+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1538022070 con 0x7f15401088f0 2024-05-14T17:01:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.268+0000 7f1536ffd700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f152c0b2990 0x7f152c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:54.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.268+0000 7f154597c700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f152c0b2990 0x7f152c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:54.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.268+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f153800f020 con 0x7f15401088f0 2024-05-14T17:01:54.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.268+0000 7f154597c700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f152c0b2990 0x7f152c0b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f153c005950 tx=0x7f153c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:54.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.272+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1538029020 con 0x7f15401088f0 2024-05-14T17:01:54.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.478+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f154002e560 con 0x7f15401088f0 2024-05-14T17:01:54.806 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:54 smithi099 ceph-mon[31851]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:54.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.479+0000 7f1536ffd700 1 -- 172.21.15.99:0/3803440310 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f15380ab5e0 con 0x7f15401088f0 2024-05-14T17:01:54.840 INFO:teuthology.orchestra.run.smithi099.stdout:236223201332 2024-05-14T17:01:54.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f152c0b2990 msgr2=0x7f152c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:54.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f152c0b2990 0x7f152c0b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f153c005950 tx=0x7f153c0058e0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 msgr2=0x7f15401998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:54.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f15401998c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15380095c0 tx=0x7f1538007a20 comp rx=0 tx=0).stop 2024-05-14T17:01:54.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 shutdown_connections 2024-05-14T17:01:54.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f152c0b2990 0x7f152c0b4e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15401088f0 0x7f15401998c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15401037f0 0x7f154019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 --2- 172.21.15.99:0/3803440310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15401041a0 0x7f154019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:54.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 >> 172.21.15.99:0/3803440310 conn(0x7f154007a6a0 msgr2=0x7f1540101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:54.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 shutdown_connections 2024-05-14T17:01:54.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:54.481+0000 7f1547be0700 1 -- 172.21.15.99:0/3803440310 wait complete. 2024-05-14T17:01:54.983 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:54 smithi193 ceph-mon[35962]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:55.468 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:55 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3803440310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T17:01:55.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.680+0000 7f4d71666700 1 -- 172.21.15.99:0/9157904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06c8c0 msgr2=0x7f4d6c108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.680+0000 7f4d71666700 1 --2- 172.21.15.99:0/9157904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06c8c0 0x7f4d6c108180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4d600099c0 tx=0x7f4d60009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 -- 172.21.15.99:0/9157904 shutdown_connections 2024-05-14T17:01:55.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 --2- 172.21.15.99:0/9157904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d6c108860 0x7f4d6c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 --2- 172.21.15.99:0/9157904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c06c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 --2- 172.21.15.99:0/9157904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06c8c0 0x7f4d6c108180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:55.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 -- 172.21.15.99:0/9157904 >> 172.21.15.99:0/9157904 conn(0x7f4d6c078490 msgr2=0x7f4d6c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 -- 172.21.15.99:0/9157904 shutdown_connections 2024-05-14T17:01:55.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 -- 172.21.15.99:0/9157904 wait complete. 2024-05-14T17:01:55.812 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:55 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3803440310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T17:01:55.822 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:55 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3803440310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T17:01:55.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 Processor -- start 2024-05-14T17:01:55.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.681+0000 7f4d71666700 1 -- start start 2024-05-14T17:01:55.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d6c06c8c0 0x7f4d6c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:55.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c108860 0x7f4d6c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:55.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c113810 con 0x7f4d6c06bf10 2024-05-14T17:01:55.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c113690 con 0x7f4d6c06c8c0 2024-05-14T17:01:55.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c113990 con 0x7f4d6c108860 2024-05-14T17:01:55.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:55.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c19bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52432/0 (socket says 172.21.15.99:52432) 2024-05-14T17:01:55.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 -- 172.21.15.99:0/958847072 learned_addr learned my addr 172.21.15.99:0/958847072 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:55.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6a7fc700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d6c06c8c0 0x7f4d6c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:55.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6b7fe700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c108860 0x7f4d6c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:56.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 -- 172.21.15.99:0/958847072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c108860 msgr2=0x7f4d6c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:56.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c108860 0x7f4d6c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 -- 172.21.15.99:0/958847072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d6c06c8c0 msgr2=0x7f4d6c19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:56.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d6c06c8c0 0x7f4d6c19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 -- 172.21.15.99:0/958847072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1a0f00 con 0x7f4d6c06bf10 2024-05-14T17:01:56.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6b7fe700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c108860 0x7f4d6c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:56.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d6affd700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c19bf40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00b530 tx=0x7f4d5c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:56.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c015020 con 0x7f4d6c06bf10 2024-05-14T17:01:56.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d5c011ce0 con 0x7f4d6c06bf10 2024-05-14T17:01:56.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c010e40 con 0x7f4d6c06bf10 2024-05-14T17:01:56.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.682+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c1a11f0 con 0x7f4d6c06bf10 2024-05-14T17:01:56.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.683+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a8a70 con 0x7f4d6c06bf10 2024-05-14T17:01:56.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.683+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6c06a230 con 0x7f4d6c06bf10 2024-05-14T17:01:56.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.685+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4d5c018020 con 0x7f4d6c06bf10 2024-05-14T17:01:56.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.685+0000 7f4d53fff700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4d540b2910 0x7f4d540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:56.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.686+0000 7f4d6a7fc700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4d540b2910 0x7f4d540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:56.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.686+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f4d5c0e20c0 con 0x7f4d6c06bf10 2024-05-14T17:01:56.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.686+0000 7f4d6a7fc700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4d540b2910 0x7f4d540b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006010 tx=0x7f4d600096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:56.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.689+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d5c01f020 con 0x7f4d6c06bf10 2024-05-14T17:01:56.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.900+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4d6c19d250 con 0x7f4d6c06bf10 2024-05-14T17:01:56.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.900+0000 7f4d53fff700 1 -- 172.21.15.99:0/958847072 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4d6c19d250 con 0x7f4d6c06bf10 2024-05-14T17:01:56.265 INFO:teuthology.orchestra.run.smithi099.stdout:34359738459 2024-05-14T17:01:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4d540b2910 msgr2=0x7f4d540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:56.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4d540b2910 0x7f4d540b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006010 tx=0x7f4d600096b0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 msgr2=0x7f4d6c19bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:56.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c19bf40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00b530 tx=0x7f4d5c00b840 comp rx=0 tx=0).stop 2024-05-14T17:01:56.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 shutdown_connections 2024-05-14T17:01:56.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f4d540b2910 0x7f4d540b4dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d6c108860 0x7f4d6c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d6c06bf10 0x7f4d6c19bf40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 --2- 172.21.15.99:0/958847072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d6c06c8c0 0x7f4d6c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:56.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 >> 172.21.15.99:0/958847072 conn(0x7f4d6c078490 msgr2=0x7f4d6c103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:56.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 shutdown_connections 2024-05-14T17:01:56.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:55.903+0000 7f4d71666700 1 -- 172.21.15.99:0/958847072 wait complete. 2024-05-14T17:01:56.624 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:56 smithi099 ceph-mon[31851]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:56.640 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:56 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/958847072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T17:01:56.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:56 smithi150 ceph-mon[36204]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:56.793 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:56 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/958847072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T17:01:56.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:56 smithi193 ceph-mon[35962]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:01:56.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:56 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/958847072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T17:01:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 -- 172.21.15.99:0/3822845413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190103850 msgr2=0x7fb190103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:57.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 --2- 172.21.15.99:0/3822845413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190103850 0x7fb190103c30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0099c0 tx=0x7fb18c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:01:57.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 -- 172.21.15.99:0/3822845413 shutdown_connections 2024-05-14T17:01:57.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 --2- 172.21.15.99:0/3822845413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 0x7fb1901104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:57.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 --2- 172.21.15.99:0/3822845413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190103850 0x7fb190103c30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:57.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 --2- 172.21.15.99:0/3822845413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190104200 0x7fb1901081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:57.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 -- 172.21.15.99:0/3822845413 >> 172.21.15.99:0/3822845413 conn(0x7fb19007a5d0 msgr2=0x7fb19007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:57.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 -- 172.21.15.99:0/3822845413 shutdown_connections 2024-05-14T17:01:57.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.184+0000 7fb197828700 1 -- 172.21.15.99:0/3822845413 wait complete. 2024-05-14T17:01:57.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 Processor -- start 2024-05-14T17:01:57.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 -- start start 2024-05-14T17:01:57.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190103850 0x7fb190075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:57.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 0x7fb190075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:57.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 0x7fb1900762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:57.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb190113760 con 0x7fb190104200 2024-05-14T17:01:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901135e0 con 0x7fb190108920 2024-05-14T17:01:57.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb197828700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1901138e0 con 0x7fb190103850 2024-05-14T17:01:57.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb194dc3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 0x7fb190075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:57.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb194dc3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 0x7fb190075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52450/0 (socket says 172.21.15.99:52450) 2024-05-14T17:01:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb194dc3700 1 -- 172.21.15.99:0/1936997423 learned_addr learned my addr 172.21.15.99:0/1936997423 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:57.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.185+0000 7fb195dc5700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 0x7fb1900762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:57.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb1955c4700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190103850 0x7fb190075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb194dc3700 1 -- 172.21.15.99:0/1936997423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190103850 msgr2=0x7fb190075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb194dc3700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190103850 0x7fb190075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:57.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb194dc3700 1 -- 172.21.15.99:0/1936997423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 msgr2=0x7fb1900762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:57.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb194dc3700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 0x7fb1900762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:57.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb194dc3700 1 -- 172.21.15.99:0/1936997423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1901a8180 con 0x7fb190104200 2024-05-14T17:01:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb195dc5700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 0x7fb1900762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:57.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb1955c4700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190103850 0x7fb190075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:57.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb194dc3700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 0x7fb190075d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb18000b530 tx=0x7fb18000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:57.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180015020 con 0x7fb190104200 2024-05-14T17:01:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1901a8470 con 0x7fb190104200 2024-05-14T17:01:57.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb180011ce0 con 0x7fb190104200 2024-05-14T17:01:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180010d20 con 0x7fb190104200 2024-05-14T17:01:57.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.186+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a89b0 con 0x7fb190104200 2024-05-14T17:01:58.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.187+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19010bb60 con 0x7fb190104200 2024-05-14T17:01:58.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.188+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb180018020 con 0x7fb190104200 2024-05-14T17:01:58.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.189+0000 7fb1867fc700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb17c0b2910 0x7fb17c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:58.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.189+0000 7fb1955c4700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb17c0b2910 0x7fb17c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:58.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.189+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fb1800e2090 con 0x7fb190104200 2024-05-14T17:01:58.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.190+0000 7fb1955c4700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb17c0b2910 0x7fb17c0b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb18c003800 tx=0x7fb18c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:58.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.193+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb18001f020 con 0x7fb190104200 2024-05-14T17:01:58.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 -- 172.21.15.99:0/3070805145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f502410fe90 msgr2=0x7f5024112280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 --2- 172.21.15.99:0/3070805145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f502410fe90 0x7f5024112280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f501c00b0a0 tx=0x7f501c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 -- 172.21.15.99:0/3070805145 shutdown_connections 2024-05-14T17:01:58.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 --2- 172.21.15.99:0/3070805145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f502410fe90 0x7f5024112280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 --2- 172.21.15.99:0/3070805145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024104490 0x7f502406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 --2- 172.21.15.99:0/3070805145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 0x7f502406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 -- 172.21.15.99:0/3070805145 >> 172.21.15.99:0/3070805145 conn(0x7f50240793f0 msgr2=0x7f5024079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 -- 172.21.15.99:0/3070805145 shutdown_connections 2024-05-14T17:01:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 -- 172.21.15.99:0/3070805145 wait complete. 2024-05-14T17:01:58.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 Processor -- start 2024-05-14T17:01:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.403+0000 7f502bf58700 1 -- start start 2024-05-14T17:01:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 0x7f502419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 0x7f502419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:58.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502410fe90 0x7f50241a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:58.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5024115940 con 0x7f5024104490 2024-05-14T17:01:58.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50241157c0 con 0x7f502410fe90 2024-05-14T17:01:58.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5024115ac0 con 0x7f502406c150 2024-05-14T17:01:58.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 0x7f502419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:58.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 0x7f502419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52470/0 (socket says 172.21.15.99:52470) 2024-05-14T17:01:58.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 -- 172.21.15.99:0/3643535592 learned_addr learned my addr 172.21.15.99:0/3643535592 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:01:58.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502a4f5700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502410fe90 0x7f50241a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:58.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f5029cf4700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 0x7f502419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:58.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 -- 172.21.15.99:0/3643535592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 msgr2=0x7f502419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 0x7f502419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 -- 172.21.15.99:0/3643535592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502410fe90 msgr2=0x7f50241a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502410fe90 0x7f50241a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 -- 172.21.15.99:0/3643535592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50241a0d50 con 0x7f5024104490 2024-05-14T17:01:58.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f5029cf4700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 0x7f502419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f50294f3700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 0x7f502419c2d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f501400b530 tx=0x7f501400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014015020 con 0x7f5024104490 2024-05-14T17:01:58.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50241a1040 con 0x7f5024104490 2024-05-14T17:01:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5014011ce0 con 0x7f5024104490 2024-05-14T17:01:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.404+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014010e40 con 0x7f5024104490 2024-05-14T17:01:58.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.405+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a88c0 con 0x7f5024104490 2024-05-14T17:01:58.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.405+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5024102a80 con 0x7f5024104490 2024-05-14T17:01:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.406+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb19010b540 con 0x7fb190104200 2024-05-14T17:01:58.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.407+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5014018020 con 0x7f5024104490 2024-05-14T17:01:58.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.408+0000 7f501affd700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f50100b2910 0x7f50100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:58.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.408+0000 7f5029cf4700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f50100b2910 0x7f50100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:58.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.408+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f50140e20c0 con 0x7f5024104490 2024-05-14T17:01:58.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.408+0000 7f5029cf4700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f50100b2910 0x7f50100b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5020005950 tx=0x7f50200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:58.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.411+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f501401f020 con 0x7f5024104490 2024-05-14T17:01:58.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.411+0000 7fb1867fc700 1 -- 172.21.15.99:0/1936997423 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1800a9d70 con 0x7fb190104200 2024-05-14T17:01:58.697 INFO:teuthology.orchestra.run.smithi099.stdout:210453397561 2024-05-14T17:01:58.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb17c0b2910 msgr2=0x7fb17c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb17c0b2910 0x7fb17c0b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb18c003800 tx=0x7fb18c009f10 comp rx=0 tx=0).stop 2024-05-14T17:01:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 msgr2=0x7fb190075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 0x7fb190075d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb18000b530 tx=0x7fb18000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 shutdown_connections 2024-05-14T17:01:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb17c0b2910 0x7fb17c0b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb190108920 0x7fb1900762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb190103850 0x7fb190075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 --2- 172.21.15.99:0/1936997423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb190104200 0x7fb190075d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:58.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 >> 172.21.15.99:0/1936997423 conn(0x7fb19007a5d0 msgr2=0x7fb190109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:58.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 shutdown_connections 2024-05-14T17:01:58.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.414+0000 7fb197828700 1 -- 172.21.15.99:0/1936997423 wait complete. 2024-05-14T17:01:58.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.627+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f50241a8b80 con 0x7f5024104490 2024-05-14T17:01:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.628+0000 7f501affd700 1 -- 172.21.15.99:0/3643535592 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5014010510 con 0x7f5024104490 2024-05-14T17:01:58.917 INFO:teuthology.orchestra.run.smithi099.stdout:77309411410 2024-05-14T17:01:58.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:58 smithi193 ceph-mon[35962]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:58.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:58 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1936997423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T17:01:58.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:58 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3643535592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T17:01:58.951 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:01:58 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1736616927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T17:01:58.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:58 smithi150 ceph-mon[36204]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:58.967 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:58 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1936997423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T17:01:59.015 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:58 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3643535592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T17:01:59.025 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:01:58 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1736616927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T17:01:59.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f50100b2910 msgr2=0x7f50100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:59.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f50100b2910 0x7f50100b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5020005950 tx=0x7f50200058e0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 msgr2=0x7f502419c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:59.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 0x7f502419c2d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f501400b530 tx=0x7f501400b840 comp rx=0 tx=0).stop 2024-05-14T17:01:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 shutdown_connections 2024-05-14T17:01:59.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f50100b2910 0x7f50100b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502410fe90 0x7f50241a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502406c150 0x7f502419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 --2- 172.21.15.99:0/3643535592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5024104490 0x7f502419c2d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 >> 172.21.15.99:0/3643535592 conn(0x7f50240793f0 msgr2=0x7f5024101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:59.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 shutdown_connections 2024-05-14T17:01:59.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:57.630+0000 7f502bf58700 1 -- 172.21.15.99:0/3643535592 wait complete. 2024-05-14T17:01:59.197 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:58 smithi099 ceph-mon[31851]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:01:59.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:58 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1936997423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T17:01:59.250 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:58 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3643535592' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T17:01:59.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:01:58 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1736616927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T17:01:59.278 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738459 for osd.0 2024-05-14T17:01:59.284 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:59.293 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411410 for osd.2 2024-05-14T17:01:59.308 DEBUG:teuthology.parallel:result is None 2024-05-14T17:01:59.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.045+0000 7f35260cc700 1 -- 172.21.15.99:0/4066281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520102430 msgr2=0x7f35201028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:59.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.045+0000 7f35260cc700 1 --2- 172.21.15.99:0/4066281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520102430 0x7f35201028b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3514009990 tx=0x7f3514009ca0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 -- 172.21.15.99:0/4066281491 shutdown_connections 2024-05-14T17:01:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 --2- 172.21.15.99:0/4066281491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3520102df0 0x7f352010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 --2- 172.21.15.99:0/4066281491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3520108510 0x7f35201088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 --2- 172.21.15.99:0/4066281491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520102430 0x7f35201028b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 -- 172.21.15.99:0/4066281491 >> 172.21.15.99:0/4066281491 conn(0x7f35200793b0 msgr2=0x7f35200797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:59.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 -- 172.21.15.99:0/4066281491 shutdown_connections 2024-05-14T17:01:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 -- 172.21.15.99:0/4066281491 wait complete. 2024-05-14T17:01:59.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.046+0000 7f35260cc700 1 Processor -- start 2024-05-14T17:01:59.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 -- start start 2024-05-14T17:01:59.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3520102430 0x7f352019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3520102df0 0x7f352019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 0x7f3520199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35201125e0 con 0x7f3520108510 2024-05-14T17:02:00.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112460 con 0x7f3520102430 2024-05-14T17:02:00.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f35260cc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112760 con 0x7f3520102df0 2024-05-14T17:02:00.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 0x7f3520199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 0x7f3520199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52498/0 (socket says 172.21.15.99:52498) 2024-05-14T17:02:00.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 -- 172.21.15.99:0/1736616927 learned_addr learned my addr 172.21.15.99:0/1736616927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351f7fe700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3520102430 0x7f352019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351effd700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3520102df0 0x7f352019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 -- 172.21.15.99:0/1736616927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3520102df0 msgr2=0x7f352019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3520102df0 0x7f352019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 -- 172.21.15.99:0/1736616927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3520102430 msgr2=0x7f352019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3520102430 0x7f352019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 -- 172.21.15.99:0/1736616927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352019a240 con 0x7f3520108510 2024-05-14T17:02:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.047+0000 7f351ffff700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 0x7f3520199ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3518007ee0 tx=0x7f351800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:00.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.048+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351800ddd0 con 0x7f3520108510 2024-05-14T17:02:00.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.048+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3518004020 con 0x7f3520108510 2024-05-14T17:02:00.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.048+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518015660 con 0x7f3520108510 2024-05-14T17:02:00.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.048+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352019a530 con 0x7f3520108510 2024-05-14T17:02:00.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.048+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201a67f0 con 0x7f3520108510 2024-05-14T17:02:00.735 INFO:teuthology.orchestra.run.smithi099.stdout:163208757316 2024-05-14T17:02:00.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.049+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352006a230 con 0x7f3520108510 2024-05-14T17:02:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.050+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f35180041d0 con 0x7f3520108510 2024-05-14T17:02:00.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.051+0000 7f351cff9700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f35080b2990 0x7f35080b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.051+0000 7f351f7fe700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f35080b2990 0x7f35080b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.051+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f35180e25b0 con 0x7f3520108510 2024-05-14T17:02:00.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.051+0000 7f351f7fe700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f35080b2990 0x7f35080b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3510005950 tx=0x7f351000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:00.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.054+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3518010070 con 0x7f3520108510 2024-05-14T17:02:00.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.269+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f352002e560 con 0x7f3520108510 2024-05-14T17:02:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.269+0000 7f351cff9700 1 -- 172.21.15.99:0/1736616927 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35180b2070 con 0x7f3520108510 2024-05-14T17:02:01.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f35080b2990 msgr2=0x7f35080b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f35080b2990 0x7f35080b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3510005950 tx=0x7f351000a300 comp rx=0 tx=0).stop 2024-05-14T17:02:01.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 msgr2=0x7f3520199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 0x7f3520199ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3518007ee0 tx=0x7f351800d3b0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 shutdown_connections 2024-05-14T17:02:01.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f35080b2990 0x7f35080b4e50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3520108510 0x7f3520199ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3520102430 0x7f352019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 --2- 172.21.15.99:0/1736616927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3520102df0 0x7f352019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 >> 172.21.15.99:0/1736616927 conn(0x7f35200793b0 msgr2=0x7f352006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:01.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 shutdown_connections 2024-05-14T17:02:01.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:01:58.271+0000 7f35260cc700 1 -- 172.21.15.99:0/1736616927 wait complete. 2024-05-14T17:02:01.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397556 got 210453397561 for osd.7 2024-05-14T17:02:01.188 DEBUG:teuthology.parallel:result is None 2024-05-14T17:02:01.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757309 got 163208757316 for osd.5 2024-05-14T17:02:01.205 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:00 smithi099 ceph-mon[31851]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:01.214 DEBUG:teuthology.parallel:result is None 2024-05-14T17:02:01.240 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:00 smithi193 ceph-mon[35962]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:01.249 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:00 smithi150 ceph-mon[36204]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:01.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201327 got 236223201332 for osd.8 2024-05-14T17:02:01.332 DEBUG:teuthology.parallel:result is None 2024-05-14T17:02:01.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 -- 172.21.15.99:0/3829410120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e4102470 msgr2=0x7f26e41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 --2- 172.21.15.99:0/3829410120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e4102470 0x7f26e41028f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009990 tx=0x7f26d0009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 -- 172.21.15.99:0/3829410120 shutdown_connections 2024-05-14T17:02:01.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 --2- 172.21.15.99:0/3829410120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26e4102e30 0x7f26e410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 --2- 172.21.15.99:0/3829410120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e41084b0 0x7f26e4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 --2- 172.21.15.99:0/3829410120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e4102470 0x7f26e41028f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 -- 172.21.15.99:0/3829410120 >> 172.21.15.99:0/3829410120 conn(0x7f26e4079410 msgr2=0x7f26e4079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:01.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 -- 172.21.15.99:0/3829410120 shutdown_connections 2024-05-14T17:02:01.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 -- 172.21.15.99:0/3829410120 wait complete. 2024-05-14T17:02:01.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 Processor -- start 2024-05-14T17:02:01.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.519+0000 7f26eafa0700 1 -- start start 2024-05-14T17:02:01.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e4102470 0x7f26e419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26e4102e30 0x7f26e419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 0x7f26e4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41125c0 con 0x7f26e41084b0 2024-05-14T17:02:01.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4112440 con 0x7f26e4102470 2024-05-14T17:02:01.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4112740 con 0x7f26e4102e30 2024-05-14T17:02:01.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 0x7f26e4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 0x7f26e4199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52522/0 (socket says 172.21.15.99:52522) 2024-05-14T17:02:01.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 -- 172.21.15.99:0/441181239 learned_addr learned my addr 172.21.15.99:0/441181239 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:01.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e8d3c700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e4102470 0x7f26e419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e3fff700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26e4102e30 0x7f26e419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 -- 172.21.15.99:0/441181239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26e4102e30 msgr2=0x7f26e419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26e4102e30 0x7f26e419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 -- 172.21.15.99:0/441181239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e4102470 msgr2=0x7f26e419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e4102470 0x7f26e419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 -- 172.21.15.99:0/441181239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e419a1b0 con 0x7f26e41084b0 2024-05-14T17:02:01.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e8d3c700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e4102470 0x7f26e419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:01.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e953d700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 0x7f26e4199a50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00ada0 tx=0x7f26dc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:01.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc017020 con 0x7f26e41084b0 2024-05-14T17:02:01.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26dc00ce30 con 0x7f26e41084b0 2024-05-14T17:02:01.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e419a4a0 con 0x7f26e41084b0 2024-05-14T17:02:01.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.520+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0054a0 con 0x7f26e41084b0 2024-05-14T17:02:01.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.521+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e41a6760 con 0x7f26e41084b0 2024-05-14T17:02:01.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.521+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e406a230 con 0x7f26e41084b0 2024-05-14T17:02:01.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.523+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f26dc00c980 con 0x7f26e41084b0 2024-05-14T17:02:01.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.523+0000 7f26e1ffb700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f26d40b2990 0x7f26d40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.523+0000 7f26e8d3c700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f26d40b2990 0x7f26d40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.523+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f26dc0e1b00 con 0x7f26e41084b0 2024-05-14T17:02:01.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.524+0000 7f26e8d3c700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f26d40b2990 0x7f26d40b4e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005ea0 tx=0x7f26d8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.528+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26dc021020 con 0x7f26e41084b0 2024-05-14T17:02:01.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:01 smithi099 ceph-mon[31851]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:01.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.739+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f26e402e560 con 0x7f26e41084b0 2024-05-14T17:02:02.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.740+0000 7f26e1ffb700 1 -- 172.21.15.99:0/441181239 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f26dc0a9760 con 0x7f26e41084b0 2024-05-14T17:02:02.031 INFO:teuthology.orchestra.run.smithi099.stdout:55834574935 2024-05-14T17:02:02.038 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:01 smithi193 ceph-mon[35962]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:02.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:01 smithi150 ceph-mon[36204]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f26d40b2990 msgr2=0x7f26d40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:02.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f26d40b2990 0x7f26d40b4e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005ea0 tx=0x7f26d8005e30 comp rx=0 tx=0).stop 2024-05-14T17:02:02.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 msgr2=0x7f26e4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:02.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 0x7f26e4199a50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00ada0 tx=0x7f26dc00c330 comp rx=0 tx=0).stop 2024-05-14T17:02:02.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 shutdown_connections 2024-05-14T17:02:02.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f26d40b2990 0x7f26d40b4e50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e41084b0 0x7f26e4199a50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e4102470 0x7f26e419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 --2- 172.21.15.99:0/441181239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26e4102e30 0x7f26e419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 >> 172.21.15.99:0/441181239 conn(0x7f26e4079410 msgr2=0x7f26e406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:02.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 shutdown_connections 2024-05-14T17:02:02.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:01.742+0000 7f26eafa0700 1 -- 172.21.15.99:0/441181239 wait complete. 2024-05-14T17:02:02.499 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574935 for osd.1 2024-05-14T17:02:02.506 DEBUG:teuthology.parallel:result is None 2024-05-14T17:02:02.659 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:02 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/441181239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T17:02:02.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 -- 172.21.15.99:0/1011626834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806c630 msgr2=0x7f37e81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:02.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 --2- 172.21.15.99:0/1011626834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806c630 0x7f37e81103d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009990 tx=0x7f37e0009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 -- 172.21.15.99:0/1011626834 shutdown_connections 2024-05-14T17:02:02.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 --2- 172.21.15.99:0/1011626834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806c630 0x7f37e81103d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 --2- 172.21.15.99:0/1011626834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e81095b0 0x7f37e8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 --2- 172.21.15.99:0/1011626834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37e806bc70 0x7f37e806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 -- 172.21.15.99:0/1011626834 >> 172.21.15.99:0/1011626834 conn(0x7f37e80793f0 msgr2=0x7f37e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:02.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 -- 172.21.15.99:0/1011626834 shutdown_connections 2024-05-14T17:02:02.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.669+0000 7f37ef651700 1 -- 172.21.15.99:0/1011626834 wait complete. 2024-05-14T17:02:02.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 Processor -- start 2024-05-14T17:02:02.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 -- start start 2024-05-14T17:02:02.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 0x7f37e8102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:02.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e806c630 0x7f37e8102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:02.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37e81095b0 0x7f37e8102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:02.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8113740 con 0x7f37e806bc70 2024-05-14T17:02:02.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81135c0 con 0x7f37e81095b0 2024-05-14T17:02:02.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ef651700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81138c0 con 0x7f37e806c630 2024-05-14T17:02:02.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 0x7f37e8102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:02.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 0x7f37e8102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52548/0 (socket says 172.21.15.99:52548) 2024-05-14T17:02:02.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 -- 172.21.15.99:0/2769519842 learned_addr learned my addr 172.21.15.99:0/2769519842 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:02.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 -- 172.21.15.99:0/2769519842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e806c630 msgr2=0x7f37e8102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e806c630 0x7f37e8102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 -- 172.21.15.99:0/2769519842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37e81095b0 msgr2=0x7f37e8102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37edbee700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37e81095b0 0x7f37e8102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:02.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37e81095b0 0x7f37e8102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.670+0000 7f37ed3ed700 1 -- 172.21.15.99:0/2769519842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81a8130 con 0x7f37e806bc70 2024-05-14T17:02:02.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.671+0000 7f37ed3ed700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 0x7f37e8102540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f37e400ba50 tx=0x7f37e400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:03.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.671+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4019070 con 0x7f37e806bc70 2024-05-14T17:02:03.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.671+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e4014440 con 0x7f37e806bc70 2024-05-14T17:02:03.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.671+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81a8420 con 0x7f37e806bc70 2024-05-14T17:02:03.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.671+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e40075d0 con 0x7f37e806bc70 2024-05-14T17:02:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.671+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a89b0 con 0x7f37e806bc70 2024-05-14T17:02:03.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.672+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e810ba90 con 0x7f37e806bc70 2024-05-14T17:02:03.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.673+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f37e401c070 con 0x7f37e806bc70 2024-05-14T17:02:03.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.674+0000 7f37de7fc700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f37d40b2990 0x7f37d40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:03.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.674+0000 7f37ecbec700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f37d40b2990 0x7f37d40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:03.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.674+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f37e40e1b60 con 0x7f37e806bc70 2024-05-14T17:02:03.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.674+0000 7f37ecbec700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f37d40b2990 0x7f37d40b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005950 tx=0x7f37d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:03.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.677+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37e40a9840 con 0x7f37e806bc70 2024-05-14T17:02:03.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:02 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/441181239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T17:02:03.190 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:02 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/441181239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T17:02:03.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.880+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f37e810b470 con 0x7f37e806bc70 2024-05-14T17:02:03.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.880+0000 7f37de7fc700 1 -- 172.21.15.99:0/2769519842 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f37e40a9140 con 0x7f37e806bc70 2024-05-14T17:02:03.217 INFO:teuthology.orchestra.run.smithi099.stdout:283467841579 2024-05-14T17:02:03.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f37d40b2990 msgr2=0x7f37d40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:03.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f37d40b2990 0x7f37d40b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005950 tx=0x7f37d80058e0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 msgr2=0x7f37e8102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 0x7f37e8102540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f37e400ba50 tx=0x7f37e400be10 comp rx=0 tx=0).stop 2024-05-14T17:02:03.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 shutdown_connections 2024-05-14T17:02:03.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f37d40b2990 0x7f37d40b4e50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37e81095b0 0x7f37e8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37e806bc70 0x7f37e8102540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 --2- 172.21.15.99:0/2769519842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37e806c630 0x7f37e8102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:03.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 >> 172.21.15.99:0/2769519842 conn(0x7f37e80793f0 msgr2=0x7f37e81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:03.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 shutdown_connections 2024-05-14T17:02:03.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:02.883+0000 7f37ef651700 1 -- 172.21.15.99:0/2769519842 wait complete. 2024-05-14T17:02:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:03 smithi099 ceph-mon[31851]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:03.536 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:03 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2769519842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T17:02:03.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841574 got 283467841579 for osd.10 2024-05-14T17:02:03.598 DEBUG:teuthology.parallel:result is None 2024-05-14T17:02:03.605 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T17:02:03.613 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:02:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:03 smithi150 ceph-mon[36204]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:03.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:03 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2769519842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T17:02:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:03 smithi193 ceph-mon[35962]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:03.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:03 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2769519842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T17:02:03.862 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:04.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:04 smithi099 ceph-mon[31851]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T17:02:04.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:04 smithi150 ceph-mon[36204]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T17:02:04.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:04 smithi193 ceph-mon[35962]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T17:02:05.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 -- 172.21.15.99:0/1314400874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094103600 msgr2=0x7f809410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:05.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 --2- 172.21.15.99:0/1314400874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094103600 0x7f809410f2e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b0a0 tx=0x7f808c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 -- 172.21.15.99:0/1314400874 shutdown_connections 2024-05-14T17:02:05.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 --2- 172.21.15.99:0/1314400874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094103600 0x7f809410f2e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 --2- 172.21.15.99:0/1314400874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80941084c0 0x7f80941088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 --2- 172.21.15.99:0/1314400874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094102c40 0x7f80941030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 -- 172.21.15.99:0/1314400874 >> 172.21.15.99:0/1314400874 conn(0x7f8094078ab0 msgr2=0x7f80940ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:05.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 -- 172.21.15.99:0/1314400874 shutdown_connections 2024-05-14T17:02:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.480+0000 7f8098d99700 1 -- 172.21.15.99:0/1314400874 wait complete. 2024-05-14T17:02:05.559 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:05 smithi099 ceph-mon[31851]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:05.586 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:05 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:05.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 Processor -- start 2024-05-14T17:02:05.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 -- start start 2024-05-14T17:02:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 0x7f8094198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:05.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094103600 0x7f8094198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:05.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80941084c0 0x7f809419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:05.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80941125f0 con 0x7f80941084c0 2024-05-14T17:02:05.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8094112470 con 0x7f8094103600 2024-05-14T17:02:05.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f8098d99700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8094112770 con 0x7f8094102c40 2024-05-14T17:02:05.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.481+0000 7f809259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 0x7f8094198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:05.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f8091d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094103600 0x7f8094198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:05.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 0x7f8094198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:50740/0 (socket says 172.21.15.99:50740) 2024-05-14T17:02:05.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 -- 172.21.15.99:0/2296377465 learned_addr learned my addr 172.21.15.99:0/2296377465 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:05.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f8092d9d700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80941084c0 0x7f809419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:05.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 -- 172.21.15.99:0/2296377465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094103600 msgr2=0x7f8094198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:05.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094103600 0x7f8094198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 -- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80941084c0 msgr2=0x7f809419d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:05.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80941084c0 0x7f809419d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:05.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 -- 172.21.15.99:0/2296377465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809419dcd0 con 0x7f8094102c40 2024-05-14T17:02:05.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f809259c700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 0x7f8094198740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f808400e950 tx=0x7f808400ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004230 con 0x7f8094102c40 2024-05-14T17:02:05.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.482+0000 7f8091d9b700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094103600 0x7f8094198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:05.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.483+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809419dfc0 con 0x7f8094102c40 2024-05-14T17:02:05.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.483+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80840043d0 con 0x7f8094102c40 2024-05-14T17:02:05.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.483+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809419e500 con 0x7f8094102c40 2024-05-14T17:02:05.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.483+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084003d60 con 0x7f8094102c40 2024-05-14T17:02:05.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.484+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8094103c50 con 0x7f8094102c40 2024-05-14T17:02:05.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.485+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f808400c7e0 con 0x7f8094102c40 2024-05-14T17:02:05.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.486+0000 7f80837fe700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f807c0b2880 0x7f807c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:05.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.486+0000 7f8091d9b700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f807c0b2880 0x7f807c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.486+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f80840e2390 con 0x7f8094102c40 2024-05-14T17:02:05.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.486+0000 7f8091d9b700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f807c0b2880 0x7f807c0b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8094107a50 tx=0x7f8088009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:05.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.491+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80840aa0c0 con 0x7f8094102c40 2024-05-14T17:02:06.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:05 smithi193 ceph-mon[35962]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:06.044 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:05 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:06.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:05 smithi150 ceph-mon[36204]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:06.077 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:05 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.721+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f809419e8d0 con 0x7f807c0b2880 2024-05-14T17:02:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.728+0000 7f80837fe700 1 -- 172.21.15.99:0/2296377465 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258809 (secure 0 0 0) 0x7f809419e8d0 con 0x7f807c0b2880 2024-05-14T17:02:06.104 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:06.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f807c0b2880 msgr2=0x7f807c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f807c0b2880 0x7f807c0b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8094107a50 tx=0x7f8088009260 comp rx=0 tx=0).stop 2024-05-14T17:02:06.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 msgr2=0x7f8094198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:06.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 0x7f8094198740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f808400e950 tx=0x7f808400ec60 comp rx=0 tx=0).stop 2024-05-14T17:02:06.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 shutdown_connections 2024-05-14T17:02:06.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f807c0b2880 0x7f807c0b4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:06.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80941084c0 0x7f809419d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8094102c40 0x7f8094198740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:06.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 --2- 172.21.15.99:0/2296377465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094103600 0x7f8094198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:06.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 >> 172.21.15.99:0/2296377465 conn(0x7f8094078ab0 msgr2=0x7f80940fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:06.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 shutdown_connections 2024-05-14T17:02:06.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:05.732+0000 7f8098d99700 1 -- 172.21.15.99:0/2296377465 wait complete. 2024-05-14T17:02:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:02:06.592 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:06 smithi099 ceph-mon[31851]: from='client.24670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:02:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:06 smithi150 ceph-mon[36204]: from='client.24670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:02:06.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:06 smithi193 ceph-mon[35962]: from='client.24670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:02:06.901 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-05-14T17:02:04.485885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":681,"num_read_kb":436,"num_write":295,"num_write_kb":2267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2952887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370016,"kb_used_data":12492,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571808,"statfs":{"total":1151940427776,"available":1151561531392,"internally_reserved":0,"allocated":12791808,"data_stored":7150689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002740"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231841+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:01:04.231841+0000","last_peered":"2024-05-14T17:01:04.231841+0000","last_clean":"2024-05-14T17:01:04.231841+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:01:04.231841+0000","last_undegraded":"2024-05-14T17:01:04.231841+0000","last_fullsized":"2024-05-14T17:01:04.231841+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.480323+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:02:02.480323+0000","last_peered":"2024-05-14T17:02:02.480323+0000","last_clean":"2024-05-14T17:02:02.480323+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:02:02.480323+0000","last_undegraded":"2024-05-14T17:02:02.480323+0000","last_fullsized":"2024-05-14T17:02:02.480323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.475683+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:02:02.475683+0000","last_peered":"2024-05-14T17:02:02.475683+0000","last_clean":"2024-05-14T17:02:02.475683+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:02:02.475683+0000","last_undegraded":"2024-05-14T17:02:02.475683+0000","last_fullsized":"2024-05-14T17:02:02.475683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485883+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:02:02.485883+0000","last_peered":"2024-05-14T17:02:02.485883+0000","last_clean":"2024-05-14T17:02:02.485883+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:02:02.485883+0000","last_undegraded":"2024-05-14T17:02:02.485883+0000","last_fullsized":"2024-05-14T17:02:02.485883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234309+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:01:04.234309+0000","last_peered":"2024-05-14T17:01:04.234309+0000","last_clean":"2024-05-14T17:01:04.234309+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:01:04.234309+0000","last_undegraded":"2024-05-14T17:01:04.234309+0000","last_fullsized":"2024-05-14T17:01:04.234309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.475901+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:02:02.475901+0000","last_peered":"2024-05-14T17:02:02.475901+0000","last_clean":"2024-05-14T17:02:02.475901+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:02:02.475901+0000","last_undegraded":"2024-05-14T17:02:02.475901+0000","last_fullsized":"2024-05-14T17:02:02.475901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234595+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:01:04.234595+0000","last_peered":"2024-05-14T17:01:04.234595+0000","last_clean":"2024-05-14T17:01:04.234595+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:01:04.234595+0000","last_undegraded":"2024-05-14T17:01:04.234595+0000","last_fullsized":"2024-05-14T17:01:04.234595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235799+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:01:04.235799+0000","last_peered":"2024-05-14T17:01:04.235799+0000","last_clean":"2024-05-14T17:01:04.235799+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:01:04.235799+0000","last_undegraded":"2024-05-14T17:01:04.235799+0000","last_fullsized":"2024-05-14T17:01:04.235799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.266368+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:01:04.266368+0000","last_peered":"2024-05-14T17:01:04.266368+0000","last_clean":"2024-05-14T17:01:04.266368+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:01:04.266368+0000","last_undegraded":"2024-05-14T17:01:04.266368+0000","last_fullsized":"2024-05-14T17:01:04.266368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.486596+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:02:02.486596+0000","last_peered":"2024-05-14T17:02:02.486596+0000","last_clean":"2024-05-14T17:02:02.486596+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:02:02.486596+0000","last_undegraded":"2024-05-14T17:02:02.486596+0000","last_fullsized":"2024-05-14T17:02:02.486596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235055+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:01:04.235055+0000","last_peered":"2024-05-14T17:01:04.235055+0000","last_clean":"2024-05-14T17:01:04.235055+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:01:04.235055+0000","last_undegraded":"2024-05-14T17:01:04.235055+0000","last_fullsized":"2024-05-14T17:01:04.235055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.466878+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:02:02.466878+0000","last_peered":"2024-05-14T17:02:02.466878+0000","last_clean":"2024-05-14T17:02:02.466878+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:02:02.466878+0000","last_undegraded":"2024-05-14T17:02:02.466878+0000","last_fullsized":"2024-05-14T17:02:02.466878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.230986+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:01:04.230986+0000","last_peered":"2024-05-14T17:01:04.230986+0000","last_clean":"2024-05-14T17:01:04.230986+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:01:04.230986+0000","last_undegraded":"2024-05-14T17:01:04.230986+0000","last_fullsized":"2024-05-14T17:01:04.230986+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.469514+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:02:02.469514+0000","last_peered":"2024-05-14T17:02:02.469514+0000","last_clean":"2024-05-14T17:02:02.469514+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:02:02.469514+0000","last_undegraded":"2024-05-14T17:02:02.469514+0000","last_fullsized":"2024-05-14T17:02:02.469514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479385+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:02:02.479385+0000","last_peered":"2024-05-14T17:02:02.479385+0000","last_clean":"2024-05-14T17:02:02.479385+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:02:02.479385+0000","last_undegraded":"2024-05-14T17:02:02.479385+0000","last_fullsized":"2024-05-14T17:02:02.479385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.265243+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:01:04.265243+0000","last_peered":"2024-05-14T17:01:04.265243+0000","last_clean":"2024-05-14T17:01:04.265243+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:01:04.265243+0000","last_undegraded":"2024-05-14T17:01:04.265243+0000","last_fullsized":"2024-05-14T17:01:04.265243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232003+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:01:04.232003+0000","last_peered":"2024-05-14T17:01:04.232003+0000","last_clean":"2024-05-14T17:01:04.232003+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:01:04.232003+0000","last_undegraded":"2024-05-14T17:01:04.232003+0000","last_fullsized":"2024-05-14T17:01:04.232003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.473852+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:02:02.473852+0000","last_peered":"2024-05-14T17:02:02.473852+0000","last_clean":"2024-05-14T17:02:02.473852+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:02:02.473852+0000","last_undegraded":"2024-05-14T17:02:02.473852+0000","last_fullsized":"2024-05-14T17:02:02.473852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234586+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:01:04.234586+0000","last_peered":"2024-05-14T17:01:04.234586+0000","last_clean":"2024-05-14T17:01:04.234586+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:01:04.234586+0000","last_undegraded":"2024-05-14T17:01:04.234586+0000","last_fullsized":"2024-05-14T17:01:04.234586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232751+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:01:04.232751+0000","last_peered":"2024-05-14T17:01:04.232751+0000","last_clean":"2024-05-14T17:01:04.232751+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:01:04.232751+0000","last_undegraded":"2024-05-14T17:01:04.232751+0000","last_fullsized":"2024-05-14T17:01:04.232751+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235800+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:01:04.235800+0000","last_peered":"2024-05-14T17:01:04.235800+0000","last_clean":"2024-05-14T17:01:04.235800+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:01:04.235800+0000","last_undegraded":"2024-05-14T17:01:04.235800+0000","last_fullsized":"2024-05-14T17:01:04.235800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.237528+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:01:04.237528+0000","last_peered":"2024-05-14T17:01:04.237528+0000","last_clean":"2024-05-14T17:01:04.237528+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:01:04.237528+0000","last_undegraded":"2024-05-14T17:01:04.237528+0000","last_fullsized":"2024-05-14T17:01:04.237528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.483688+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:02:02.483688+0000","last_peered":"2024-05-14T17:02:02.483688+0000","last_clean":"2024-05-14T17:02:02.483688+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:02:02.483688+0000","last_undegraded":"2024-05-14T17:02:02.483688+0000","last_fullsized":"2024-05-14T17:02:02.483688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234755+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:01:04.234755+0000","last_peered":"2024-05-14T17:01:04.234755+0000","last_clean":"2024-05-14T17:01:04.234755+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:01:04.234755+0000","last_undegraded":"2024-05-14T17:01:04.234755+0000","last_fullsized":"2024-05-14T17:01:04.234755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.265667+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:01:04.265667+0000","last_peered":"2024-05-14T17:01:04.265667+0000","last_clean":"2024-05-14T17:01:04.265667+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:01:04.265667+0000","last_undegraded":"2024-05-14T17:01:04.265667+0000","last_fullsized":"2024-05-14T17:01:04.265667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234863+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:01:04.234863+0000","last_peered":"2024-05-14T17:01:04.234863+0000","last_clean":"2024-05-14T17:01:04.234863+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:01:04.234863+0000","last_undegraded":"2024-05-14T17:01:04.234863+0000","last_fullsized":"2024-05-14T17:01:04.234863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.482626+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:02:02.482626+0000","last_peered":"2024-05-14T17:02:02.482626+0000","last_clean":"2024-05-14T17:02:02.482626+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:02:02.482626+0000","last_undegraded":"2024-05-14T17:02:02.482626+0000","last_fullsized":"2024-05-14T17:02:02.482626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:03.362358+0000","last_change":"2024-05-14T17:01:04.271215+0000","last_active":"2024-05-14T17:02:03.362358+0000","last_peered":"2024-05-14T17:02:03.362358+0000","last_clean":"2024-05-14T17:02:03.362358+0000","last_became_active":"2024-05-14T17:01:04.267186+0000","last_became_peered":"2024-05-14T17:01:04.267186+0000","last_unstale":"2024-05-14T17:02:03.362358+0000","last_undegraded":"2024-05-14T17:02:03.362358+0000","last_fullsized":"2024-05-14T17:02:03.362358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:01.465730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.241079+0000","last_change":"2024-05-14T17:01:04.241079+0000","last_active":"2024-05-14T17:01:04.241079+0000","last_peered":"2024-05-14T17:01:04.241079+0000","last_clean":"2024-05-14T17:01:04.241079+0000","last_became_active":"2024-05-14T17:01:04.240698+0000","last_became_peered":"2024-05-14T17:01:04.240698+0000","last_unstale":"2024-05-14T17:01:04.241079+0000","last_undegraded":"2024-05-14T17:01:04.241079+0000","last_fullsized":"2024-05-14T17:01:04.241079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:10.589991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232250+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:01:04.232250+0000","last_peered":"2024-05-14T17:01:04.232250+0000","last_clean":"2024-05-14T17:01:04.232250+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:01:04.232250+0000","last_undegraded":"2024-05-14T17:01:04.232250+0000","last_fullsized":"2024-05-14T17:01:04.232250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.474500+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:02:02.474500+0000","last_peered":"2024-05-14T17:02:02.474500+0000","last_clean":"2024-05-14T17:02:02.474500+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:02:02.474500+0000","last_undegraded":"2024-05-14T17:02:02.474500+0000","last_fullsized":"2024-05-14T17:02:02.474500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.230791+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:01:04.230791+0000","last_peered":"2024-05-14T17:01:04.230791+0000","last_clean":"2024-05-14T17:01:04.230791+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:01:04.230791+0000","last_undegraded":"2024-05-14T17:01:04.230791+0000","last_fullsized":"2024-05-14T17:01:04.230791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479410+0000","last_change":"2024-05-14T17:01:04.236554+0000","last_active":"2024-05-14T17:02:02.479410+0000","last_peered":"2024-05-14T17:02:02.479410+0000","last_clean":"2024-05-14T17:02:02.479410+0000","last_became_active":"2024-05-14T17:01:04.236203+0000","last_became_peered":"2024-05-14T17:01:04.236203+0000","last_unstale":"2024-05-14T17:02:02.479410+0000","last_undegraded":"2024-05-14T17:02:02.479410+0000","last_fullsized":"2024-05-14T17:02:02.479410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:34.421246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238240+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:01:04.238240+0000","last_peered":"2024-05-14T17:01:04.238240+0000","last_clean":"2024-05-14T17:01:04.238240+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:01:04.238240+0000","last_undegraded":"2024-05-14T17:01:04.238240+0000","last_fullsized":"2024-05-14T17:01:04.238240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479874+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:02:02.479874+0000","last_peered":"2024-05-14T17:02:02.479874+0000","last_clean":"2024-05-14T17:02:02.479874+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:02:02.479874+0000","last_undegraded":"2024-05-14T17:02:02.479874+0000","last_fullsized":"2024-05-14T17:02:02.479874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.437443+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:02:02.437443+0000","last_peered":"2024-05-14T17:02:02.437443+0000","last_clean":"2024-05-14T17:02:02.437443+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:02:02.437443+0000","last_undegraded":"2024-05-14T17:02:02.437443+0000","last_fullsized":"2024-05-14T17:02:02.437443+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.266269+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:01:04.266269+0000","last_peered":"2024-05-14T17:01:04.266269+0000","last_clean":"2024-05-14T17:01:04.266269+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:01:04.266269+0000","last_undegraded":"2024-05-14T17:01:04.266269+0000","last_fullsized":"2024-05-14T17:01:04.266269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235061+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:01:04.235061+0000","last_peered":"2024-05-14T17:01:04.235061+0000","last_clean":"2024-05-14T17:01:04.235061+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:01:04.235061+0000","last_undegraded":"2024-05-14T17:01:04.235061+0000","last_fullsized":"2024-05-14T17:01:04.235061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.477546+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:02:02.477546+0000","last_peered":"2024-05-14T17:02:02.477546+0000","last_clean":"2024-05-14T17:02:02.477546+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:02:02.477546+0000","last_undegraded":"2024-05-14T17:02:02.477546+0000","last_fullsized":"2024-05-14T17:02:02.477546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238581+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:01:04.238581+0000","last_peered":"2024-05-14T17:01:04.238581+0000","last_clean":"2024-05-14T17:01:04.238581+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:01:04.238581+0000","last_undegraded":"2024-05-14T17:01:04.238581+0000","last_fullsized":"2024-05-14T17:01:04.238581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.265860+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:01:04.265860+0000","last_peered":"2024-05-14T17:01:04.265860+0000","last_clean":"2024-05-14T17:01:04.265860+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:01:04.265860+0000","last_undegraded":"2024-05-14T17:01:04.265860+0000","last_fullsized":"2024-05-14T17:01:04.265860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":259,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:03.362346+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:02:03.362346+0000","last_peered":"2024-05-14T17:02:03.362346+0000","last_clean":"2024-05-14T17:02:03.362346+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:02:03.362346+0000","last_undegraded":"2024-05-14T17:02:03.362346+0000","last_fullsized":"2024-05-14T17:02:03.362346+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":202,"num_read_kb":160,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234563+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:01:04.234563+0000","last_peered":"2024-05-14T17:01:04.234563+0000","last_clean":"2024-05-14T17:01:04.234563+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:01:04.234563+0000","last_undegraded":"2024-05-14T17:01:04.234563+0000","last_fullsized":"2024-05-14T17:01:04.234563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.249901+0000","last_change":"2024-05-14T17:01:04.249537+0000","last_active":"2024-05-14T17:01:04.249901+0000","last_peered":"2024-05-14T17:01:04.249901+0000","last_clean":"2024-05-14T17:01:04.249901+0000","last_became_active":"2024-05-14T17:01:04.239963+0000","last_became_peered":"2024-05-14T17:01:04.239963+0000","last_unstale":"2024-05-14T17:01:04.249901+0000","last_undegraded":"2024-05-14T17:01:04.249901+0000","last_fullsized":"2024-05-14T17:01:04.249901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:00:51.798116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.230768+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:01:04.230768+0000","last_peered":"2024-05-14T17:01:04.230768+0000","last_clean":"2024-05-14T17:01:04.230768+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:01:04.230768+0000","last_undegraded":"2024-05-14T17:01:04.230768+0000","last_fullsized":"2024-05-14T17:01:04.230768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238991+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:01:04.238991+0000","last_peered":"2024-05-14T17:01:04.238991+0000","last_clean":"2024-05-14T17:01:04.238991+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:01:04.238991+0000","last_undegraded":"2024-05-14T17:01:04.238991+0000","last_fullsized":"2024-05-14T17:01:04.238991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234860+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:01:04.234860+0000","last_peered":"2024-05-14T17:01:04.234860+0000","last_clean":"2024-05-14T17:01:04.234860+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:01:04.234860+0000","last_undegraded":"2024-05-14T17:01:04.234860+0000","last_fullsized":"2024-05-14T17:01:04.234860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.285208+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:02:02.285208+0000","last_peered":"2024-05-14T17:02:02.285208+0000","last_clean":"2024-05-14T17:02:02.285208+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:02:02.285208+0000","last_undegraded":"2024-05-14T17:02:02.285208+0000","last_fullsized":"2024-05-14T17:02:02.285208+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234768+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:01:04.234768+0000","last_peered":"2024-05-14T17:01:04.234768+0000","last_clean":"2024-05-14T17:01:04.234768+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:01:04.234768+0000","last_undegraded":"2024-05-14T17:01:04.234768+0000","last_fullsized":"2024-05-14T17:01:04.234768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234073+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:01:04.234073+0000","last_peered":"2024-05-14T17:01:04.234073+0000","last_clean":"2024-05-14T17:01:04.234073+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:01:04.234073+0000","last_undegraded":"2024-05-14T17:01:04.234073+0000","last_fullsized":"2024-05-14T17:01:04.234073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479077+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:02:02.479077+0000","last_peered":"2024-05-14T17:02:02.479077+0000","last_clean":"2024-05-14T17:02:02.479077+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:02:02.479077+0000","last_undegraded":"2024-05-14T17:02:02.479077+0000","last_fullsized":"2024-05-14T17:02:02.479077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234556+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:01:04.234556+0000","last_peered":"2024-05-14T17:01:04.234556+0000","last_clean":"2024-05-14T17:01:04.234556+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:01:04.234556+0000","last_undegraded":"2024-05-14T17:01:04.234556+0000","last_fullsized":"2024-05-14T17:01:04.234556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231673+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:01:04.231673+0000","last_peered":"2024-05-14T17:01:04.231673+0000","last_clean":"2024-05-14T17:01:04.231673+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:01:04.231673+0000","last_undegraded":"2024-05-14T17:01:04.231673+0000","last_fullsized":"2024-05-14T17:01:04.231673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232360+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:01:04.232360+0000","last_peered":"2024-05-14T17:01:04.232360+0000","last_clean":"2024-05-14T17:01:04.232360+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:01:04.232360+0000","last_undegraded":"2024-05-14T17:01:04.232360+0000","last_fullsized":"2024-05-14T17:01:04.232360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484552+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:02:02.484552+0000","last_peered":"2024-05-14T17:02:02.484552+0000","last_clean":"2024-05-14T17:02:02.484552+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:02:02.484552+0000","last_undegraded":"2024-05-14T17:02:02.484552+0000","last_fullsized":"2024-05-14T17:02:02.484552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234741+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:01:04.234741+0000","last_peered":"2024-05-14T17:01:04.234741+0000","last_clean":"2024-05-14T17:01:04.234741+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:01:04.234741+0000","last_undegraded":"2024-05-14T17:01:04.234741+0000","last_fullsized":"2024-05-14T17:01:04.234741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.471176+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:02:02.471176+0000","last_peered":"2024-05-14T17:02:02.471176+0000","last_clean":"2024-05-14T17:02:02.471176+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:02:02.471176+0000","last_undegraded":"2024-05-14T17:02:02.471176+0000","last_fullsized":"2024-05-14T17:02:02.471176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.460909+0000","last_change":"2024-05-14T17:01:04.237414+0000","last_active":"2024-05-14T17:02:02.460909+0000","last_peered":"2024-05-14T17:02:02.460909+0000","last_clean":"2024-05-14T17:02:02.460909+0000","last_became_active":"2024-05-14T17:01:04.237043+0000","last_became_peered":"2024-05-14T17:01:04.237043+0000","last_unstale":"2024-05-14T17:02:02.460909+0000","last_undegraded":"2024-05-14T17:02:02.460909+0000","last_fullsized":"2024-05-14T17:02:02.460909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:15:33.303302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":123,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:59.280077+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:01:59.280077+0000","last_peered":"2024-05-14T17:01:59.280077+0000","last_clean":"2024-05-14T17:01:59.280077+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:01:59.280077+0000","last_undegraded":"2024-05-14T17:01:59.280077+0000","last_fullsized":"2024-05-14T17:01:59.280077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.237157+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:01:04.237157+0000","last_peered":"2024-05-14T17:01:04.237157+0000","last_clean":"2024-05-14T17:01:04.237157+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:01:04.237157+0000","last_undegraded":"2024-05-14T17:01:04.237157+0000","last_fullsized":"2024-05-14T17:01:04.237157+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.266299+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:01:04.266299+0000","last_peered":"2024-05-14T17:01:04.266299+0000","last_clean":"2024-05-14T17:01:04.266299+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:01:04.266299+0000","last_undegraded":"2024-05-14T17:01:04.266299+0000","last_fullsized":"2024-05-14T17:01:04.266299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234978+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:01:04.234978+0000","last_peered":"2024-05-14T17:01:04.234978+0000","last_clean":"2024-05-14T17:01:04.234978+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:01:04.234978+0000","last_undegraded":"2024-05-14T17:01:04.234978+0000","last_fullsized":"2024-05-14T17:01:04.234978+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.230647+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:01:04.230647+0000","last_peered":"2024-05-14T17:01:04.230647+0000","last_clean":"2024-05-14T17:01:04.230647+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:01:04.230647+0000","last_undegraded":"2024-05-14T17:01:04.230647+0000","last_fullsized":"2024-05-14T17:01:04.230647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235041+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:01:04.235041+0000","last_peered":"2024-05-14T17:01:04.235041+0000","last_clean":"2024-05-14T17:01:04.235041+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:01:04.235041+0000","last_undegraded":"2024-05-14T17:01:04.235041+0000","last_fullsized":"2024-05-14T17:01:04.235041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234556+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:01:04.234556+0000","last_peered":"2024-05-14T17:01:04.234556+0000","last_clean":"2024-05-14T17:01:04.234556+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:01:04.234556+0000","last_undegraded":"2024-05-14T17:01:04.234556+0000","last_fullsized":"2024-05-14T17:01:04.234556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.230953+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:01:04.230953+0000","last_peered":"2024-05-14T17:01:04.230953+0000","last_clean":"2024-05-14T17:01:04.230953+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:01:04.230953+0000","last_undegraded":"2024-05-14T17:01:04.230953+0000","last_fullsized":"2024-05-14T17:01:04.230953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232975+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:01:04.232975+0000","last_peered":"2024-05-14T17:01:04.232975+0000","last_clean":"2024-05-14T17:01:04.232975+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:01:04.232975+0000","last_undegraded":"2024-05-14T17:01:04.232975+0000","last_fullsized":"2024-05-14T17:01:04.232975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.237386+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:01:04.237386+0000","last_peered":"2024-05-14T17:01:04.237386+0000","last_clean":"2024-05-14T17:01:04.237386+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:01:04.237386+0000","last_undegraded":"2024-05-14T17:01:04.237386+0000","last_fullsized":"2024-05-14T17:01:04.237386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.266563+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:01:04.266563+0000","last_peered":"2024-05-14T17:01:04.266563+0000","last_clean":"2024-05-14T17:01:04.266563+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:01:04.266563+0000","last_undegraded":"2024-05-14T17:01:04.266563+0000","last_fullsized":"2024-05-14T17:01:04.266563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.478350+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:02:02.478350+0000","last_peered":"2024-05-14T17:02:02.478350+0000","last_clean":"2024-05-14T17:02:02.478350+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:02:02.478350+0000","last_undegraded":"2024-05-14T17:02:02.478350+0000","last_fullsized":"2024-05-14T17:02:02.478350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.265237+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:01:04.265237+0000","last_peered":"2024-05-14T17:01:04.265237+0000","last_clean":"2024-05-14T17:01:04.265237+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:01:04.265237+0000","last_undegraded":"2024-05-14T17:01:04.265237+0000","last_fullsized":"2024-05-14T17:01:04.265237+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234038+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:01:04.234038+0000","last_peered":"2024-05-14T17:01:04.234038+0000","last_clean":"2024-05-14T17:01:04.234038+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:01:04.234038+0000","last_undegraded":"2024-05-14T17:01:04.234038+0000","last_fullsized":"2024-05-14T17:01:04.234038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.236556+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:01:04.236556+0000","last_peered":"2024-05-14T17:01:04.236556+0000","last_clean":"2024-05-14T17:01:04.236556+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:01:04.236556+0000","last_undegraded":"2024-05-14T17:01:04.236556+0000","last_fullsized":"2024-05-14T17:01:04.236556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231387+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:01:04.231387+0000","last_peered":"2024-05-14T17:01:04.231387+0000","last_clean":"2024-05-14T17:01:04.231387+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:01:04.231387+0000","last_undegraded":"2024-05-14T17:01:04.231387+0000","last_fullsized":"2024-05-14T17:01:04.231387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234804+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:01:04.234804+0000","last_peered":"2024-05-14T17:01:04.234804+0000","last_clean":"2024-05-14T17:01:04.234804+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:01:04.234804+0000","last_undegraded":"2024-05-14T17:01:04.234804+0000","last_fullsized":"2024-05-14T17:01:04.234804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.237006+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:01:04.237006+0000","last_peered":"2024-05-14T17:01:04.237006+0000","last_clean":"2024-05-14T17:01:04.237006+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:01:04.237006+0000","last_undegraded":"2024-05-14T17:01:04.237006+0000","last_fullsized":"2024-05-14T17:01:04.237006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.243937+0000","last_change":"2024-05-14T17:01:04.243512+0000","last_active":"2024-05-14T17:01:04.243937+0000","last_peered":"2024-05-14T17:01:04.243937+0000","last_clean":"2024-05-14T17:01:04.243937+0000","last_became_active":"2024-05-14T17:01:04.237118+0000","last_became_peered":"2024-05-14T17:01:04.237118+0000","last_unstale":"2024-05-14T17:01:04.243937+0000","last_undegraded":"2024-05-14T17:01:04.243937+0000","last_fullsized":"2024-05-14T17:01:04.243937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:19:16.524481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234781+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:01:04.234781+0000","last_peered":"2024-05-14T17:01:04.234781+0000","last_clean":"2024-05-14T17:01:04.234781+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:01:04.234781+0000","last_undegraded":"2024-05-14T17:01:04.234781+0000","last_fullsized":"2024-05-14T17:01:04.234781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234366+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:01:04.234366+0000","last_peered":"2024-05-14T17:01:04.234366+0000","last_clean":"2024-05-14T17:01:04.234366+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:01:04.234366+0000","last_undegraded":"2024-05-14T17:01:04.234366+0000","last_fullsized":"2024-05-14T17:01:04.234366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235169+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:01:04.235169+0000","last_peered":"2024-05-14T17:01:04.235169+0000","last_clean":"2024-05-14T17:01:04.235169+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:01:04.235169+0000","last_undegraded":"2024-05-14T17:01:04.235169+0000","last_fullsized":"2024-05-14T17:01:04.235169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238377+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:01:04.238377+0000","last_peered":"2024-05-14T17:01:04.238377+0000","last_clean":"2024-05-14T17:01:04.238377+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:01:04.238377+0000","last_undegraded":"2024-05-14T17:01:04.238377+0000","last_fullsized":"2024-05-14T17:01:04.238377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234925+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:01:04.234925+0000","last_peered":"2024-05-14T17:01:04.234925+0000","last_clean":"2024-05-14T17:01:04.234925+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:01:04.234925+0000","last_undegraded":"2024-05-14T17:01:04.234925+0000","last_fullsized":"2024-05-14T17:01:04.234925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231125+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:01:04.231125+0000","last_peered":"2024-05-14T17:01:04.231125+0000","last_clean":"2024-05-14T17:01:04.231125+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:01:04.231125+0000","last_undegraded":"2024-05-14T17:01:04.231125+0000","last_fullsized":"2024-05-14T17:01:04.231125+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235045+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:01:04.235045+0000","last_peered":"2024-05-14T17:01:04.235045+0000","last_clean":"2024-05-14T17:01:04.235045+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:01:04.235045+0000","last_undegraded":"2024-05-14T17:01:04.235045+0000","last_fullsized":"2024-05-14T17:01:04.235045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231823+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:01:04.231823+0000","last_peered":"2024-05-14T17:01:04.231823+0000","last_clean":"2024-05-14T17:01:04.231823+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:01:04.231823+0000","last_undegraded":"2024-05-14T17:01:04.231823+0000","last_fullsized":"2024-05-14T17:01:04.231823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.233869+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:01:04.233869+0000","last_peered":"2024-05-14T17:01:04.233869+0000","last_clean":"2024-05-14T17:01:04.233869+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:01:04.233869+0000","last_undegraded":"2024-05-14T17:01:04.233869+0000","last_fullsized":"2024-05-14T17:01:04.233869+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234716+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:01:04.234716+0000","last_peered":"2024-05-14T17:01:04.234716+0000","last_clean":"2024-05-14T17:01:04.234716+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:01:04.234716+0000","last_undegraded":"2024-05-14T17:01:04.234716+0000","last_fullsized":"2024-05-14T17:01:04.234716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485200+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:02:02.485200+0000","last_peered":"2024-05-14T17:02:02.485200+0000","last_clean":"2024-05-14T17:02:02.485200+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:02:02.485200+0000","last_undegraded":"2024-05-14T17:02:02.485200+0000","last_fullsized":"2024-05-14T17:02:02.485200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231043+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:01:04.231043+0000","last_peered":"2024-05-14T17:01:04.231043+0000","last_clean":"2024-05-14T17:01:04.231043+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:01:04.231043+0000","last_undegraded":"2024-05-14T17:01:04.231043+0000","last_fullsized":"2024-05-14T17:01:04.231043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231596+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:01:04.231596+0000","last_peered":"2024-05-14T17:01:04.231596+0000","last_clean":"2024-05-14T17:01:04.231596+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:01:04.231596+0000","last_undegraded":"2024-05-14T17:01:04.231596+0000","last_fullsized":"2024-05-14T17:01:04.231596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234730+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:01:04.234730+0000","last_peered":"2024-05-14T17:01:04.234730+0000","last_clean":"2024-05-14T17:01:04.234730+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:01:04.234730+0000","last_undegraded":"2024-05-14T17:01:04.234730+0000","last_fullsized":"2024-05-14T17:01:04.234730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238941+0000","last_change":"2024-05-14T17:01:04.238941+0000","last_active":"2024-05-14T17:01:04.238941+0000","last_peered":"2024-05-14T17:01:04.238941+0000","last_clean":"2024-05-14T17:01:04.238941+0000","last_became_active":"2024-05-14T17:01:04.238526+0000","last_became_peered":"2024-05-14T17:01:04.238526+0000","last_unstale":"2024-05-14T17:01:04.238941+0000","last_undegraded":"2024-05-14T17:01:04.238941+0000","last_fullsized":"2024-05-14T17:01:04.238941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:41:40.081665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.476610+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:02:02.476610+0000","last_peered":"2024-05-14T17:02:02.476610+0000","last_clean":"2024-05-14T17:02:02.476610+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:02:02.476610+0000","last_undegraded":"2024-05-14T17:02:02.476610+0000","last_fullsized":"2024-05-14T17:02:02.476610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.236435+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:01:04.236435+0000","last_peered":"2024-05-14T17:01:04.236435+0000","last_clean":"2024-05-14T17:01:04.236435+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:01:04.236435+0000","last_undegraded":"2024-05-14T17:01:04.236435+0000","last_fullsized":"2024-05-14T17:01:04.236435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.265714+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:01:04.265714+0000","last_peered":"2024-05-14T17:01:04.265714+0000","last_clean":"2024-05-14T17:01:04.265714+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:01:04.265714+0000","last_undegraded":"2024-05-14T17:01:04.265714+0000","last_fullsized":"2024-05-14T17:01:04.265714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231092+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:01:04.231092+0000","last_peered":"2024-05-14T17:01:04.231092+0000","last_clean":"2024-05-14T17:01:04.231092+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:01:04.231092+0000","last_undegraded":"2024-05-14T17:01:04.231092+0000","last_fullsized":"2024-05-14T17:01:04.231092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.239386+0000","last_change":"2024-05-14T17:01:04.239386+0000","last_active":"2024-05-14T17:01:04.239386+0000","last_peered":"2024-05-14T17:01:04.239386+0000","last_clean":"2024-05-14T17:01:04.239386+0000","last_became_active":"2024-05-14T17:01:04.239053+0000","last_became_peered":"2024-05-14T17:01:04.239053+0000","last_unstale":"2024-05-14T17:01:04.239386+0000","last_undegraded":"2024-05-14T17:01:04.239386+0000","last_fullsized":"2024-05-14T17:01:04.239386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:49.304566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":600,"num_read_kb":367,"num_write":170,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1047,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":432725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1084768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201334,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":432847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1495040,"data_stored":1024777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":888832,"data_stored":432316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":492456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":433587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":493150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":432627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36808,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708344,"statfs":{"total":95995035648,"available":95957344256,"internally_reserved":0,"allocated":1507328,"data_stored":1024602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:02:06.966 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:02:07.248 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:07.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:07 smithi150 ceph-mon[36204]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T17:02:08.812 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:08.930 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:08.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:08.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:07 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:08.995 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:07 smithi193 ceph-mon[35962]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T17:02:09.034 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:07 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.116 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:07 smithi099 ceph-mon[31851]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T17:02:09.159 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.167 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.174 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.183 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:07 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 -- 172.21.15.99:0/1420821913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c101f00 msgr2=0x7f5f5c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:09.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 --2- 172.21.15.99:0/1420821913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c101f00 0x7f5f5c102380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009990 tx=0x7f5f44009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 -- 172.21.15.99:0/1420821913 shutdown_connections 2024-05-14T17:02:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 --2- 172.21.15.99:0/1420821913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f5c1028c0 0x7f5f5c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 --2- 172.21.15.99:0/1420821913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c108620 0x7f5f5c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 --2- 172.21.15.99:0/1420821913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c101f00 0x7f5f5c102380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 -- 172.21.15.99:0/1420821913 >> 172.21.15.99:0/1420821913 conn(0x7f5f5c077780 msgr2=0x7f5f5c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:09.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 -- 172.21.15.99:0/1420821913 shutdown_connections 2024-05-14T17:02:09.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.949+0000 7f5f6091b700 1 -- 172.21.15.99:0/1420821913 wait complete. 2024-05-14T17:02:09.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 Processor -- start 2024-05-14T17:02:09.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 -- start start 2024-05-14T17:02:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c101f00 0x7f5f5c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f5c1028c0 0x7f5f5c103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 0x7f5f5c103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1137b0 con 0x7f5f5c108620 2024-05-14T17:02:09.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c113630 con 0x7f5f5c101f00 2024-05-14T17:02:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f6091b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c113930 con 0x7f5f5c1028c0 2024-05-14T17:02:09.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.950+0000 7f5f5ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 0x7f5f5c103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:09.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 0x7f5f5c103b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60038/0 (socket says 172.21.15.99:60038) 2024-05-14T17:02:09.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 -- 172.21.15.99:0/139352821 learned_addr learned my addr 172.21.15.99:0/139352821 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:09.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5a59c700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c101f00 0x7f5f5c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:09.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f59d9b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f5c1028c0 0x7f5f5c103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:09.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 -- 172.21.15.99:0/139352821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f5c1028c0 msgr2=0x7f5f5c103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:09.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f5c1028c0 0x7f5f5c103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 -- 172.21.15.99:0/139352821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c101f00 msgr2=0x7f5f5c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c101f00 0x7f5f5c1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 -- 172.21.15.99:0/139352821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c1a81f0 con 0x7f5f5c108620 2024-05-14T17:02:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5a59c700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c101f00 0x7f5f5c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:09.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f5ad9d700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 0x7f5f5c103b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b970 tx=0x7f5f5400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:09.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400d860 con 0x7f5f5c108620 2024-05-14T17:02:09.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c1a84e0 con 0x7f5f5c108620 2024-05-14T17:02:09.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5400e430 con 0x7f5f5c108620 2024-05-14T17:02:09.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54017660 con 0x7f5f5c108620 2024-05-14T17:02:09.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.951+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c1a89f0 con 0x7f5f5c108620 2024-05-14T17:02:09.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.952+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c06a230 con 0x7f5f5c108620 2024-05-14T17:02:09.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.954+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5f5400da00 con 0x7f5f5c108620 2024-05-14T17:02:09.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.954+0000 7f5f537fe700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5f480b2880 0x7f5f480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:09.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.954+0000 7f5f5a59c700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5f480b2880 0x7f5f480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:09.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.954+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f5f540e21e0 con 0x7f5f5c108620 2024-05-14T17:02:09.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.955+0000 7f5f5a59c700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5f480b2880 0x7f5f480b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c078ce0 tx=0x7f5f4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:08.958+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5f54013030 con 0x7f5f5c108620 2024-05-14T17:02:09.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.161+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f5c063980 con 0x7f5f480b2880 2024-05-14T17:02:09.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.168+0000 7f5f537fe700 1 -- 172.21.15.99:0/139352821 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258941 (secure 0 0 0) 0x7f5f5c063980 con 0x7f5f480b2880 2024-05-14T17:02:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:09.770 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:09.846 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:09.855 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.864 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.873 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.885 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:09.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5f480b2880 msgr2=0x7f5f480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:09.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5f480b2880 0x7f5f480b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c078ce0 tx=0x7f5f4c00b410 comp rx=0 tx=0).stop 2024-05-14T17:02:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 msgr2=0x7f5f5c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:09.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 0x7f5f5c103b60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b970 tx=0x7f5f5400bc80 comp rx=0 tx=0).stop 2024-05-14T17:02:09.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 shutdown_connections 2024-05-14T17:02:09.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5f480b2880 0x7f5f480b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:09.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f5c108620 0x7f5f5c103b60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f5c101f00 0x7f5f5c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:10.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 --2- 172.21.15.99:0/139352821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f5c1028c0 0x7f5f5c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:10.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 >> 172.21.15.99:0/139352821 conn(0x7f5f5c077780 msgr2=0x7f5f5c100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:10.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 shutdown_connections 2024-05-14T17:02:10.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:09.171+0000 7f5f6091b700 1 -- 172.21.15.99:0/139352821 wait complete. 2024-05-14T17:02:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:02:10.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:10.086 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:10.094 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:10.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:10.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:02:10.177 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-14T17:02:08.486990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":686,"num_read_kb":441,"num_write":295,"num_write_kb":2267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2952887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":738,"ondisk_log_size":738,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369964,"kb_used_data":12492,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571860,"statfs":{"total":1151940427776,"available":1151561584640,"internally_reserved":0,"allocated":12791808,"data_stored":7151328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002973"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.483987+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:02:02.483987+0000","last_peered":"2024-05-14T17:02:02.483987+0000","last_clean":"2024-05-14T17:02:02.483987+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:02:02.483987+0000","last_undegraded":"2024-05-14T17:02:02.483987+0000","last_fullsized":"2024-05-14T17:02:02.483987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.480323+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:02:02.480323+0000","last_peered":"2024-05-14T17:02:02.480323+0000","last_clean":"2024-05-14T17:02:02.480323+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:02:02.480323+0000","last_undegraded":"2024-05-14T17:02:02.480323+0000","last_fullsized":"2024-05-14T17:02:02.480323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.475683+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:02:02.475683+0000","last_peered":"2024-05-14T17:02:02.475683+0000","last_clean":"2024-05-14T17:02:02.475683+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:02:02.475683+0000","last_undegraded":"2024-05-14T17:02:02.475683+0000","last_fullsized":"2024-05-14T17:02:02.475683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485883+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:02:02.485883+0000","last_peered":"2024-05-14T17:02:02.485883+0000","last_clean":"2024-05-14T17:02:02.485883+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:02:02.485883+0000","last_undegraded":"2024-05-14T17:02:02.485883+0000","last_fullsized":"2024-05-14T17:02:02.485883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.478258+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:02:02.478258+0000","last_peered":"2024-05-14T17:02:02.478258+0000","last_clean":"2024-05-14T17:02:02.478258+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:02:02.478258+0000","last_undegraded":"2024-05-14T17:02:02.478258+0000","last_fullsized":"2024-05-14T17:02:02.478258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.475901+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:02:02.475901+0000","last_peered":"2024-05-14T17:02:02.475901+0000","last_clean":"2024-05-14T17:02:02.475901+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:02:02.475901+0000","last_undegraded":"2024-05-14T17:02:02.475901+0000","last_fullsized":"2024-05-14T17:02:02.475901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.478689+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:02:02.478689+0000","last_peered":"2024-05-14T17:02:02.478689+0000","last_clean":"2024-05-14T17:02:02.478689+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:02:02.478689+0000","last_undegraded":"2024-05-14T17:02:02.478689+0000","last_fullsized":"2024-05-14T17:02:02.478689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.473595+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:02:02.473595+0000","last_peered":"2024-05-14T17:02:02.473595+0000","last_clean":"2024-05-14T17:02:02.473595+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:02:02.473595+0000","last_undegraded":"2024-05-14T17:02:02.473595+0000","last_fullsized":"2024-05-14T17:02:02.473595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.481924+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:02:02.481924+0000","last_peered":"2024-05-14T17:02:02.481924+0000","last_clean":"2024-05-14T17:02:02.481924+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:02:02.481924+0000","last_undegraded":"2024-05-14T17:02:02.481924+0000","last_fullsized":"2024-05-14T17:02:02.481924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.486596+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:02:02.486596+0000","last_peered":"2024-05-14T17:02:02.486596+0000","last_clean":"2024-05-14T17:02:02.486596+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:02:02.486596+0000","last_undegraded":"2024-05-14T17:02:02.486596+0000","last_fullsized":"2024-05-14T17:02:02.486596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484199+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:02:02.484199+0000","last_peered":"2024-05-14T17:02:02.484199+0000","last_clean":"2024-05-14T17:02:02.484199+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:02:02.484199+0000","last_undegraded":"2024-05-14T17:02:02.484199+0000","last_fullsized":"2024-05-14T17:02:02.484199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.466878+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:02:02.466878+0000","last_peered":"2024-05-14T17:02:02.466878+0000","last_clean":"2024-05-14T17:02:02.466878+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:02:02.466878+0000","last_undegraded":"2024-05-14T17:02:02.466878+0000","last_fullsized":"2024-05-14T17:02:02.466878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.486210+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:02:02.486210+0000","last_peered":"2024-05-14T17:02:02.486210+0000","last_clean":"2024-05-14T17:02:02.486210+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:02:02.486210+0000","last_undegraded":"2024-05-14T17:02:02.486210+0000","last_fullsized":"2024-05-14T17:02:02.486210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.469514+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:02:02.469514+0000","last_peered":"2024-05-14T17:02:02.469514+0000","last_clean":"2024-05-14T17:02:02.469514+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:02:02.469514+0000","last_undegraded":"2024-05-14T17:02:02.469514+0000","last_fullsized":"2024-05-14T17:02:02.469514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479385+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:02:02.479385+0000","last_peered":"2024-05-14T17:02:02.479385+0000","last_clean":"2024-05-14T17:02:02.479385+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:02:02.479385+0000","last_undegraded":"2024-05-14T17:02:02.479385+0000","last_fullsized":"2024-05-14T17:02:02.479385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484884+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:02:02.484884+0000","last_peered":"2024-05-14T17:02:02.484884+0000","last_clean":"2024-05-14T17:02:02.484884+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:02:02.484884+0000","last_undegraded":"2024-05-14T17:02:02.484884+0000","last_fullsized":"2024-05-14T17:02:02.484884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484388+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:02:02.484388+0000","last_peered":"2024-05-14T17:02:02.484388+0000","last_clean":"2024-05-14T17:02:02.484388+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:02:02.484388+0000","last_undegraded":"2024-05-14T17:02:02.484388+0000","last_fullsized":"2024-05-14T17:02:02.484388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.473852+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:02:02.473852+0000","last_peered":"2024-05-14T17:02:02.473852+0000","last_clean":"2024-05-14T17:02:02.473852+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:02:02.473852+0000","last_undegraded":"2024-05-14T17:02:02.473852+0000","last_fullsized":"2024-05-14T17:02:02.473852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484515+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:02:02.484515+0000","last_peered":"2024-05-14T17:02:02.484515+0000","last_clean":"2024-05-14T17:02:02.484515+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:02:02.484515+0000","last_undegraded":"2024-05-14T17:02:02.484515+0000","last_fullsized":"2024-05-14T17:02:02.484515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232751+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:01:04.232751+0000","last_peered":"2024-05-14T17:01:04.232751+0000","last_clean":"2024-05-14T17:01:04.232751+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:01:04.232751+0000","last_undegraded":"2024-05-14T17:01:04.232751+0000","last_fullsized":"2024-05-14T17:01:04.232751+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.469661+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:02:02.469661+0000","last_peered":"2024-05-14T17:02:02.469661+0000","last_clean":"2024-05-14T17:02:02.469661+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:02:02.469661+0000","last_undegraded":"2024-05-14T17:02:02.469661+0000","last_fullsized":"2024-05-14T17:02:02.469661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.237528+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:01:04.237528+0000","last_peered":"2024-05-14T17:01:04.237528+0000","last_clean":"2024-05-14T17:01:04.237528+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:01:04.237528+0000","last_undegraded":"2024-05-14T17:01:04.237528+0000","last_fullsized":"2024-05-14T17:01:04.237528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.483688+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:02:02.483688+0000","last_peered":"2024-05-14T17:02:02.483688+0000","last_clean":"2024-05-14T17:02:02.483688+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:02:02.483688+0000","last_undegraded":"2024-05-14T17:02:02.483688+0000","last_fullsized":"2024-05-14T17:02:02.483688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234755+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:01:04.234755+0000","last_peered":"2024-05-14T17:01:04.234755+0000","last_clean":"2024-05-14T17:01:04.234755+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:01:04.234755+0000","last_undegraded":"2024-05-14T17:01:04.234755+0000","last_fullsized":"2024-05-14T17:01:04.234755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.480199+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:02:02.480199+0000","last_peered":"2024-05-14T17:02:02.480199+0000","last_clean":"2024-05-14T17:02:02.480199+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:02:02.480199+0000","last_undegraded":"2024-05-14T17:02:02.480199+0000","last_fullsized":"2024-05-14T17:02:02.480199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234863+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:01:04.234863+0000","last_peered":"2024-05-14T17:01:04.234863+0000","last_clean":"2024-05-14T17:01:04.234863+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:01:04.234863+0000","last_undegraded":"2024-05-14T17:01:04.234863+0000","last_fullsized":"2024-05-14T17:01:04.234863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.482626+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:02:02.482626+0000","last_peered":"2024-05-14T17:02:02.482626+0000","last_clean":"2024-05-14T17:02:02.482626+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:02:02.482626+0000","last_undegraded":"2024-05-14T17:02:02.482626+0000","last_fullsized":"2024-05-14T17:02:02.482626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:03.362358+0000","last_change":"2024-05-14T17:01:04.271215+0000","last_active":"2024-05-14T17:02:03.362358+0000","last_peered":"2024-05-14T17:02:03.362358+0000","last_clean":"2024-05-14T17:02:03.362358+0000","last_became_active":"2024-05-14T17:01:04.267186+0000","last_became_peered":"2024-05-14T17:01:04.267186+0000","last_unstale":"2024-05-14T17:02:03.362358+0000","last_undegraded":"2024-05-14T17:02:03.362358+0000","last_fullsized":"2024-05-14T17:02:03.362358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:01.465730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.470197+0000","last_change":"2024-05-14T17:01:04.241079+0000","last_active":"2024-05-14T17:02:02.470197+0000","last_peered":"2024-05-14T17:02:02.470197+0000","last_clean":"2024-05-14T17:02:02.470197+0000","last_became_active":"2024-05-14T17:01:04.240698+0000","last_became_peered":"2024-05-14T17:01:04.240698+0000","last_unstale":"2024-05-14T17:02:02.470197+0000","last_undegraded":"2024-05-14T17:02:02.470197+0000","last_fullsized":"2024-05-14T17:02:02.470197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:10.589991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232250+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:01:04.232250+0000","last_peered":"2024-05-14T17:01:04.232250+0000","last_clean":"2024-05-14T17:01:04.232250+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:01:04.232250+0000","last_undegraded":"2024-05-14T17:01:04.232250+0000","last_fullsized":"2024-05-14T17:01:04.232250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.474500+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:02:02.474500+0000","last_peered":"2024-05-14T17:02:02.474500+0000","last_clean":"2024-05-14T17:02:02.474500+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:02:02.474500+0000","last_undegraded":"2024-05-14T17:02:02.474500+0000","last_fullsized":"2024-05-14T17:02:02.474500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.230791+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:01:04.230791+0000","last_peered":"2024-05-14T17:01:04.230791+0000","last_clean":"2024-05-14T17:01:04.230791+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:01:04.230791+0000","last_undegraded":"2024-05-14T17:01:04.230791+0000","last_fullsized":"2024-05-14T17:01:04.230791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479410+0000","last_change":"2024-05-14T17:01:04.236554+0000","last_active":"2024-05-14T17:02:02.479410+0000","last_peered":"2024-05-14T17:02:02.479410+0000","last_clean":"2024-05-14T17:02:02.479410+0000","last_became_active":"2024-05-14T17:01:04.236203+0000","last_became_peered":"2024-05-14T17:01:04.236203+0000","last_unstale":"2024-05-14T17:02:02.479410+0000","last_undegraded":"2024-05-14T17:02:02.479410+0000","last_fullsized":"2024-05-14T17:02:02.479410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:34.421246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238240+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:01:04.238240+0000","last_peered":"2024-05-14T17:01:04.238240+0000","last_clean":"2024-05-14T17:01:04.238240+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:01:04.238240+0000","last_undegraded":"2024-05-14T17:01:04.238240+0000","last_fullsized":"2024-05-14T17:01:04.238240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479874+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:02:02.479874+0000","last_peered":"2024-05-14T17:02:02.479874+0000","last_clean":"2024-05-14T17:02:02.479874+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:02:02.479874+0000","last_undegraded":"2024-05-14T17:02:02.479874+0000","last_fullsized":"2024-05-14T17:02:02.479874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.437443+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:02:02.437443+0000","last_peered":"2024-05-14T17:02:02.437443+0000","last_clean":"2024-05-14T17:02:02.437443+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:02:02.437443+0000","last_undegraded":"2024-05-14T17:02:02.437443+0000","last_fullsized":"2024-05-14T17:02:02.437443+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.472569+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:02:02.472569+0000","last_peered":"2024-05-14T17:02:02.472569+0000","last_clean":"2024-05-14T17:02:02.472569+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:02:02.472569+0000","last_undegraded":"2024-05-14T17:02:02.472569+0000","last_fullsized":"2024-05-14T17:02:02.472569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235061+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:01:04.235061+0000","last_peered":"2024-05-14T17:01:04.235061+0000","last_clean":"2024-05-14T17:01:04.235061+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:01:04.235061+0000","last_undegraded":"2024-05-14T17:01:04.235061+0000","last_fullsized":"2024-05-14T17:01:04.235061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.477546+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:02:02.477546+0000","last_peered":"2024-05-14T17:02:02.477546+0000","last_clean":"2024-05-14T17:02:02.477546+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:02:02.477546+0000","last_undegraded":"2024-05-14T17:02:02.477546+0000","last_fullsized":"2024-05-14T17:02:02.477546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238581+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:01:04.238581+0000","last_peered":"2024-05-14T17:01:04.238581+0000","last_clean":"2024-05-14T17:01:04.238581+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:01:04.238581+0000","last_undegraded":"2024-05-14T17:01:04.238581+0000","last_fullsized":"2024-05-14T17:01:04.238581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.481116+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:02:02.481116+0000","last_peered":"2024-05-14T17:02:02.481116+0000","last_clean":"2024-05-14T17:02:02.481116+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:02:02.481116+0000","last_undegraded":"2024-05-14T17:02:02.481116+0000","last_fullsized":"2024-05-14T17:02:02.481116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":259,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:03.362346+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:02:03.362346+0000","last_peered":"2024-05-14T17:02:03.362346+0000","last_clean":"2024-05-14T17:02:03.362346+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:02:03.362346+0000","last_undegraded":"2024-05-14T17:02:03.362346+0000","last_fullsized":"2024-05-14T17:02:03.362346+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":202,"num_read_kb":160,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.475371+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:02:02.475371+0000","last_peered":"2024-05-14T17:02:02.475371+0000","last_clean":"2024-05-14T17:02:02.475371+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:02:02.475371+0000","last_undegraded":"2024-05-14T17:02:02.475371+0000","last_fullsized":"2024-05-14T17:02:02.475371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.249901+0000","last_change":"2024-05-14T17:01:04.249537+0000","last_active":"2024-05-14T17:01:04.249901+0000","last_peered":"2024-05-14T17:01:04.249901+0000","last_clean":"2024-05-14T17:01:04.249901+0000","last_became_active":"2024-05-14T17:01:04.239963+0000","last_became_peered":"2024-05-14T17:01:04.239963+0000","last_unstale":"2024-05-14T17:01:04.249901+0000","last_undegraded":"2024-05-14T17:01:04.249901+0000","last_fullsized":"2024-05-14T17:01:04.249901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:00:51.798116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.486980+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:02:02.486980+0000","last_peered":"2024-05-14T17:02:02.486980+0000","last_clean":"2024-05-14T17:02:02.486980+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:02:02.486980+0000","last_undegraded":"2024-05-14T17:02:02.486980+0000","last_fullsized":"2024-05-14T17:02:02.486980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238991+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:01:04.238991+0000","last_peered":"2024-05-14T17:01:04.238991+0000","last_clean":"2024-05-14T17:01:04.238991+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:01:04.238991+0000","last_undegraded":"2024-05-14T17:01:04.238991+0000","last_fullsized":"2024-05-14T17:01:04.238991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.486956+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:02:02.486956+0000","last_peered":"2024-05-14T17:02:02.486956+0000","last_clean":"2024-05-14T17:02:02.486956+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:02:02.486956+0000","last_undegraded":"2024-05-14T17:02:02.486956+0000","last_fullsized":"2024-05-14T17:02:02.486956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'3","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.285208+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:02:02.285208+0000","last_peered":"2024-05-14T17:02:02.285208+0000","last_clean":"2024-05-14T17:02:02.285208+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:02:02.285208+0000","last_undegraded":"2024-05-14T17:02:02.285208+0000","last_fullsized":"2024-05-14T17:02:02.285208+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479023+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:02:02.479023+0000","last_peered":"2024-05-14T17:02:02.479023+0000","last_clean":"2024-05-14T17:02:02.479023+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:02:02.479023+0000","last_undegraded":"2024-05-14T17:02:02.479023+0000","last_fullsized":"2024-05-14T17:02:02.479023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234073+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:01:04.234073+0000","last_peered":"2024-05-14T17:01:04.234073+0000","last_clean":"2024-05-14T17:01:04.234073+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:01:04.234073+0000","last_undegraded":"2024-05-14T17:01:04.234073+0000","last_fullsized":"2024-05-14T17:01:04.234073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479077+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:02:02.479077+0000","last_peered":"2024-05-14T17:02:02.479077+0000","last_clean":"2024-05-14T17:02:02.479077+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:02:02.479077+0000","last_undegraded":"2024-05-14T17:02:02.479077+0000","last_fullsized":"2024-05-14T17:02:02.479077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234556+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:01:04.234556+0000","last_peered":"2024-05-14T17:01:04.234556+0000","last_clean":"2024-05-14T17:01:04.234556+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:01:04.234556+0000","last_undegraded":"2024-05-14T17:01:04.234556+0000","last_fullsized":"2024-05-14T17:01:04.234556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.478117+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:02:02.478117+0000","last_peered":"2024-05-14T17:02:02.478117+0000","last_clean":"2024-05-14T17:02:02.478117+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:02:02.478117+0000","last_undegraded":"2024-05-14T17:02:02.478117+0000","last_fullsized":"2024-05-14T17:02:02.478117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232360+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:01:04.232360+0000","last_peered":"2024-05-14T17:01:04.232360+0000","last_clean":"2024-05-14T17:01:04.232360+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:01:04.232360+0000","last_undegraded":"2024-05-14T17:01:04.232360+0000","last_fullsized":"2024-05-14T17:01:04.232360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484552+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:02:02.484552+0000","last_peered":"2024-05-14T17:02:02.484552+0000","last_clean":"2024-05-14T17:02:02.484552+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:02:02.484552+0000","last_undegraded":"2024-05-14T17:02:02.484552+0000","last_fullsized":"2024-05-14T17:02:02.484552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234741+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:01:04.234741+0000","last_peered":"2024-05-14T17:01:04.234741+0000","last_clean":"2024-05-14T17:01:04.234741+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:01:04.234741+0000","last_undegraded":"2024-05-14T17:01:04.234741+0000","last_fullsized":"2024-05-14T17:01:04.234741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.471176+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:02:02.471176+0000","last_peered":"2024-05-14T17:02:02.471176+0000","last_clean":"2024-05-14T17:02:02.471176+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:02:02.471176+0000","last_undegraded":"2024-05-14T17:02:02.471176+0000","last_fullsized":"2024-05-14T17:02:02.471176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.460909+0000","last_change":"2024-05-14T17:01:04.237414+0000","last_active":"2024-05-14T17:02:02.460909+0000","last_peered":"2024-05-14T17:02:02.460909+0000","last_clean":"2024-05-14T17:02:02.460909+0000","last_became_active":"2024-05-14T17:01:04.237043+0000","last_became_peered":"2024-05-14T17:01:04.237043+0000","last_unstale":"2024-05-14T17:02:02.460909+0000","last_undegraded":"2024-05-14T17:02:02.460909+0000","last_fullsized":"2024-05-14T17:02:02.460909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:15:33.303302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":128,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:04.287823+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:02:04.287823+0000","last_peered":"2024-05-14T17:02:04.287823+0000","last_clean":"2024-05-14T17:02:04.287823+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:02:04.287823+0000","last_undegraded":"2024-05-14T17:02:04.287823+0000","last_fullsized":"2024-05-14T17:02:04.287823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.472884+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:02:02.472884+0000","last_peered":"2024-05-14T17:02:02.472884+0000","last_clean":"2024-05-14T17:02:02.472884+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:02:02.472884+0000","last_undegraded":"2024-05-14T17:02:02.472884+0000","last_fullsized":"2024-05-14T17:02:02.472884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.266299+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:01:04.266299+0000","last_peered":"2024-05-14T17:01:04.266299+0000","last_clean":"2024-05-14T17:01:04.266299+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:01:04.266299+0000","last_undegraded":"2024-05-14T17:01:04.266299+0000","last_fullsized":"2024-05-14T17:01:04.266299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234978+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:01:04.234978+0000","last_peered":"2024-05-14T17:01:04.234978+0000","last_clean":"2024-05-14T17:01:04.234978+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:01:04.234978+0000","last_undegraded":"2024-05-14T17:01:04.234978+0000","last_fullsized":"2024-05-14T17:01:04.234978+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485948+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:02:02.485948+0000","last_peered":"2024-05-14T17:02:02.485948+0000","last_clean":"2024-05-14T17:02:02.485948+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:02:02.485948+0000","last_undegraded":"2024-05-14T17:02:02.485948+0000","last_fullsized":"2024-05-14T17:02:02.485948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.235041+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:01:04.235041+0000","last_peered":"2024-05-14T17:01:04.235041+0000","last_clean":"2024-05-14T17:01:04.235041+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:01:04.235041+0000","last_undegraded":"2024-05-14T17:01:04.235041+0000","last_fullsized":"2024-05-14T17:01:04.235041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234556+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:01:04.234556+0000","last_peered":"2024-05-14T17:01:04.234556+0000","last_clean":"2024-05-14T17:01:04.234556+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:01:04.234556+0000","last_undegraded":"2024-05-14T17:01:04.234556+0000","last_fullsized":"2024-05-14T17:01:04.234556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479761+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:02:02.479761+0000","last_peered":"2024-05-14T17:02:02.479761+0000","last_clean":"2024-05-14T17:02:02.479761+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:02:02.479761+0000","last_undegraded":"2024-05-14T17:02:02.479761+0000","last_fullsized":"2024-05-14T17:02:02.479761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.232975+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:01:04.232975+0000","last_peered":"2024-05-14T17:01:04.232975+0000","last_clean":"2024-05-14T17:01:04.232975+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:01:04.232975+0000","last_undegraded":"2024-05-14T17:01:04.232975+0000","last_fullsized":"2024-05-14T17:01:04.232975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.483732+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:02:02.483732+0000","last_peered":"2024-05-14T17:02:02.483732+0000","last_clean":"2024-05-14T17:02:02.483732+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:02:02.483732+0000","last_undegraded":"2024-05-14T17:02:02.483732+0000","last_fullsized":"2024-05-14T17:02:02.483732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.266563+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:01:04.266563+0000","last_peered":"2024-05-14T17:01:04.266563+0000","last_clean":"2024-05-14T17:01:04.266563+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:01:04.266563+0000","last_undegraded":"2024-05-14T17:01:04.266563+0000","last_fullsized":"2024-05-14T17:01:04.266563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.478350+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:02:02.478350+0000","last_peered":"2024-05-14T17:02:02.478350+0000","last_clean":"2024-05-14T17:02:02.478350+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:02:02.478350+0000","last_undegraded":"2024-05-14T17:02:02.478350+0000","last_fullsized":"2024-05-14T17:02:02.478350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.265237+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:01:04.265237+0000","last_peered":"2024-05-14T17:01:04.265237+0000","last_clean":"2024-05-14T17:01:04.265237+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:01:04.265237+0000","last_undegraded":"2024-05-14T17:01:04.265237+0000","last_fullsized":"2024-05-14T17:01:04.265237+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.442216+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:02:02.442216+0000","last_peered":"2024-05-14T17:02:02.442216+0000","last_clean":"2024-05-14T17:02:02.442216+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:02:02.442216+0000","last_undegraded":"2024-05-14T17:02:02.442216+0000","last_fullsized":"2024-05-14T17:02:02.442216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.236556+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:01:04.236556+0000","last_peered":"2024-05-14T17:01:04.236556+0000","last_clean":"2024-05-14T17:01:04.236556+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:01:04.236556+0000","last_undegraded":"2024-05-14T17:01:04.236556+0000","last_fullsized":"2024-05-14T17:01:04.236556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485171+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:02:02.485171+0000","last_peered":"2024-05-14T17:02:02.485171+0000","last_clean":"2024-05-14T17:02:02.485171+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:02:02.485171+0000","last_undegraded":"2024-05-14T17:02:02.485171+0000","last_fullsized":"2024-05-14T17:02:02.485171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234804+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:01:04.234804+0000","last_peered":"2024-05-14T17:01:04.234804+0000","last_clean":"2024-05-14T17:01:04.234804+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:01:04.234804+0000","last_undegraded":"2024-05-14T17:01:04.234804+0000","last_fullsized":"2024-05-14T17:01:04.234804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.471827+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:02:02.471827+0000","last_peered":"2024-05-14T17:02:02.471827+0000","last_clean":"2024-05-14T17:02:02.471827+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:02:02.471827+0000","last_undegraded":"2024-05-14T17:02:02.471827+0000","last_fullsized":"2024-05-14T17:02:02.471827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.243937+0000","last_change":"2024-05-14T17:01:04.243512+0000","last_active":"2024-05-14T17:01:04.243937+0000","last_peered":"2024-05-14T17:01:04.243937+0000","last_clean":"2024-05-14T17:01:04.243937+0000","last_became_active":"2024-05-14T17:01:04.237118+0000","last_became_peered":"2024-05-14T17:01:04.237118+0000","last_unstale":"2024-05-14T17:01:04.243937+0000","last_undegraded":"2024-05-14T17:01:04.243937+0000","last_fullsized":"2024-05-14T17:01:04.243937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:19:16.524481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.486558+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:02:02.486558+0000","last_peered":"2024-05-14T17:02:02.486558+0000","last_clean":"2024-05-14T17:02:02.486558+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:02:02.486558+0000","last_undegraded":"2024-05-14T17:02:02.486558+0000","last_fullsized":"2024-05-14T17:02:02.486558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.234366+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:01:04.234366+0000","last_peered":"2024-05-14T17:01:04.234366+0000","last_clean":"2024-05-14T17:01:04.234366+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:01:04.234366+0000","last_undegraded":"2024-05-14T17:01:04.234366+0000","last_fullsized":"2024-05-14T17:01:04.234366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484384+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:02:02.484384+0000","last_peered":"2024-05-14T17:02:02.484384+0000","last_clean":"2024-05-14T17:02:02.484384+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:02:02.484384+0000","last_undegraded":"2024-05-14T17:02:02.484384+0000","last_fullsized":"2024-05-14T17:02:02.484384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.238377+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:01:04.238377+0000","last_peered":"2024-05-14T17:01:04.238377+0000","last_clean":"2024-05-14T17:01:04.238377+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:01:04.238377+0000","last_undegraded":"2024-05-14T17:01:04.238377+0000","last_fullsized":"2024-05-14T17:01:04.238377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.478555+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:02:02.478555+0000","last_peered":"2024-05-14T17:02:02.478555+0000","last_clean":"2024-05-14T17:02:02.478555+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:02:02.478555+0000","last_undegraded":"2024-05-14T17:02:02.478555+0000","last_fullsized":"2024-05-14T17:02:02.478555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:01:04.231125+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:01:04.231125+0000","last_peered":"2024-05-14T17:01:04.231125+0000","last_clean":"2024-05-14T17:01:04.231125+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:01:04.231125+0000","last_undegraded":"2024-05-14T17:01:04.231125+0000","last_fullsized":"2024-05-14T17:01:04.231125+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.479731+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:02:02.479731+0000","last_peered":"2024-05-14T17:02:02.479731+0000","last_clean":"2024-05-14T17:02:02.479731+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:02:02.479731+0000","last_undegraded":"2024-05-14T17:02:02.479731+0000","last_fullsized":"2024-05-14T17:02:02.479731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485273+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:02:02.485273+0000","last_peered":"2024-05-14T17:02:02.485273+0000","last_clean":"2024-05-14T17:02:02.485273+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:02:02.485273+0000","last_undegraded":"2024-05-14T17:02:02.485273+0000","last_fullsized":"2024-05-14T17:02:02.485273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.483612+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:02:02.483612+0000","last_peered":"2024-05-14T17:02:02.483612+0000","last_clean":"2024-05-14T17:02:02.483612+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:02:02.483612+0000","last_undegraded":"2024-05-14T17:02:02.483612+0000","last_fullsized":"2024-05-14T17:02:02.483612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.483079+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:02:02.483079+0000","last_peered":"2024-05-14T17:02:02.483079+0000","last_clean":"2024-05-14T17:02:02.483079+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:02:02.483079+0000","last_undegraded":"2024-05-14T17:02:02.483079+0000","last_fullsized":"2024-05-14T17:02:02.483079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485200+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:02:02.485200+0000","last_peered":"2024-05-14T17:02:02.485200+0000","last_clean":"2024-05-14T17:02:02.485200+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:02:02.485200+0000","last_undegraded":"2024-05-14T17:02:02.485200+0000","last_fullsized":"2024-05-14T17:02:02.485200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485974+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:02:02.485974+0000","last_peered":"2024-05-14T17:02:02.485974+0000","last_clean":"2024-05-14T17:02:02.485974+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:02:02.485974+0000","last_undegraded":"2024-05-14T17:02:02.485974+0000","last_fullsized":"2024-05-14T17:02:02.485974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.480297+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:02:02.480297+0000","last_peered":"2024-05-14T17:02:02.480297+0000","last_clean":"2024-05-14T17:02:02.480297+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:02:02.480297+0000","last_undegraded":"2024-05-14T17:02:02.480297+0000","last_fullsized":"2024-05-14T17:02:02.480297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.484835+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:02:02.484835+0000","last_peered":"2024-05-14T17:02:02.484835+0000","last_clean":"2024-05-14T17:02:02.484835+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:02:02.484835+0000","last_undegraded":"2024-05-14T17:02:02.484835+0000","last_fullsized":"2024-05-14T17:02:02.484835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.464358+0000","last_change":"2024-05-14T17:01:04.238941+0000","last_active":"2024-05-14T17:02:02.464358+0000","last_peered":"2024-05-14T17:02:02.464358+0000","last_clean":"2024-05-14T17:02:02.464358+0000","last_became_active":"2024-05-14T17:01:04.238526+0000","last_became_peered":"2024-05-14T17:01:04.238526+0000","last_unstale":"2024-05-14T17:02:02.464358+0000","last_undegraded":"2024-05-14T17:02:02.464358+0000","last_fullsized":"2024-05-14T17:02:02.464358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:41:40.081665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.476610+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:02:02.476610+0000","last_peered":"2024-05-14T17:02:02.476610+0000","last_clean":"2024-05-14T17:02:02.476610+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:02:02.476610+0000","last_undegraded":"2024-05-14T17:02:02.476610+0000","last_fullsized":"2024-05-14T17:02:02.476610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.485973+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:02:02.485973+0000","last_peered":"2024-05-14T17:02:02.485973+0000","last_clean":"2024-05-14T17:02:02.485973+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:02:02.485973+0000","last_undegraded":"2024-05-14T17:02:02.485973+0000","last_fullsized":"2024-05-14T17:02:02.485973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.482936+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:02:02.482936+0000","last_peered":"2024-05-14T17:02:02.482936+0000","last_clean":"2024-05-14T17:02:02.482936+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:02:02.482936+0000","last_undegraded":"2024-05-14T17:02:02.482936+0000","last_fullsized":"2024-05-14T17:02:02.482936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.487191+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:02:02.487191+0000","last_peered":"2024-05-14T17:02:02.487191+0000","last_clean":"2024-05-14T17:02:02.487191+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:02:02.487191+0000","last_undegraded":"2024-05-14T17:02:02.487191+0000","last_fullsized":"2024-05-14T17:02:02.487191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-14T17:02:02.481990+0000","last_change":"2024-05-14T17:01:04.239386+0000","last_active":"2024-05-14T17:02:02.481990+0000","last_peered":"2024-05-14T17:02:02.481990+0000","last_clean":"2024-05-14T17:02:02.481990+0000","last_became_active":"2024-05-14T17:01:04.239053+0000","last_became_peered":"2024-05-14T17:01:04.239053+0000","last_unstale":"2024-05-14T17:02:02.481990+0000","last_undegraded":"2024-05-14T17:02:02.481990+0000","last_fullsized":"2024-05-14T17:02:02.481990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:49.304566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":605,"num_read_kb":372,"num_write":170,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1047,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1084768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":432847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1024918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561089,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":432316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":492705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":909312,"data_stored":433587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":493399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36808,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708344,"statfs":{"total":95995035648,"available":95957344256,"internally_reserved":0,"allocated":1507328,"data_stored":1024602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:02:10.221 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T17:02:10.230 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-14T17:02:10.239 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-14T17:02:10.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-14T17:02:10.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:10 smithi099 ceph-mon[31851]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T17:02:10.272 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:10 smithi099 ceph-mon[31851]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:02:10.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:10 smithi193 ceph-mon[35962]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T17:02:10.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:10 smithi193 ceph-mon[35962]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:02:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:10 smithi150 ceph-mon[36204]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T17:02:10.488 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:10 smithi150 ceph-mon[36204]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:02:10.500 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:12.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:12 smithi099 ceph-mon[31851]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:12.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 -- 172.21.15.99:0/3545197851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41041a0 msgr2=0x7f5eb41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3545197851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41041a0 0x7f5eb41081c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40099c0 tx=0x7f5ea4009cd0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 -- 172.21.15.99:0/3545197851 shutdown_connections 2024-05-14T17:02:12.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3545197851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eb41088f0 0x7f5eb4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3545197851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 0x7f5eb4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3545197851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41041a0 0x7f5eb41081c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 -- 172.21.15.99:0/3545197851 >> 172.21.15.99:0/3545197851 conn(0x7f5eb407a6a0 msgr2=0x7f5eb407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:12.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 -- 172.21.15.99:0/3545197851 shutdown_connections 2024-05-14T17:02:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.231+0000 7f5ebc278700 1 -- 172.21.15.99:0/3545197851 wait complete. 2024-05-14T17:02:12.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 Processor -- start 2024-05-14T17:02:12.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 -- start start 2024-05-14T17:02:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 0x7f5eb419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eb41041a0 0x7f5eb419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 0x7f5eb41978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41115f0 con 0x7f5eb41088f0 2024-05-14T17:02:12.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4111470 con 0x7f5eb41037f0 2024-05-14T17:02:12.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5ebc278700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4111770 con 0x7f5eb41041a0 2024-05-14T17:02:12.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 0x7f5eb41978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 0x7f5eb41978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60050/0 (socket says 172.21.15.99:60050) 2024-05-14T17:02:12.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 -- 172.21.15.99:0/3065689926 learned_addr learned my addr 172.21.15.99:0/3065689926 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eb9813700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eb41041a0 0x7f5eb419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba014700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 0x7f5eb419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 -- 172.21.15.99:0/3065689926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eb41041a0 msgr2=0x7f5eb419dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eb41041a0 0x7f5eb419dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 -- 172.21.15.99:0/3065689926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 msgr2=0x7f5eb419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 0x7f5eb419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba815700 1 -- 172.21.15.99:0/3065689926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb4198030 con 0x7f5eb41088f0 2024-05-14T17:02:12.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.232+0000 7f5eba014700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 0x7f5eb419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.233+0000 7f5eba815700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 0x7f5eb41978d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00b970 tx=0x7f5eac00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:12.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.233+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac019070 con 0x7f5eb41088f0 2024-05-14T17:02:12.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.233+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eac014440 con 0x7f5eb41088f0 2024-05-14T17:02:12.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.233+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb4198320 con 0x7f5eb41088f0 2024-05-14T17:02:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.233+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00ce60 con 0x7f5eb41088f0 2024-05-14T17:02:12.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.233+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb41a45e0 con 0x7f5eb41088f0 2024-05-14T17:02:12.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.234+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb406a230 con 0x7f5eb41088f0 2024-05-14T17:02:12.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.236+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5eac01c070 con 0x7f5eb41088f0 2024-05-14T17:02:12.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.236+0000 7f5eab7fe700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5ea00b6c90 0x7f5ea00b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.237+0000 7f5eba014700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5ea00b6c90 0x7f5ea00b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.237+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f5eac0e1ae0 con 0x7f5eb41088f0 2024-05-14T17:02:12.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.237+0000 7f5eba014700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5ea00b6c90 0x7f5ea00b9150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4198a10 tx=0x7f5eb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:12.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.240+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5eac0a9810 con 0x7f5eb41088f0 2024-05-14T17:02:12.727 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:12 smithi150 ceph-mon[36204]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:12.765 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:12 smithi193 ceph-mon[35962]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:12.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.487+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5eb410bb30 con 0x7f5eb41088f0 2024-05-14T17:02:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.488+0000 7f5eab7fe700 1 -- 172.21.15.99:0/3065689926 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5eac0a9110 con 0x7f5eb41088f0 2024-05-14T17:02:12.820 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:12.827 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T17:02:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5ea00b6c90 msgr2=0x7f5ea00b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5ea00b6c90 0x7f5ea00b9150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4198a10 tx=0x7f5eb0008040 comp rx=0 tx=0).stop 2024-05-14T17:02:12.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 msgr2=0x7f5eb41978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 0x7f5eb41978d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00b970 tx=0x7f5eac00bd30 comp rx=0 tx=0).stop 2024-05-14T17:02:12.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 shutdown_connections 2024-05-14T17:02:12.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5ea00b6c90 0x7f5ea00b9150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb41088f0 0x7f5eb41978d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eb41037f0 0x7f5eb419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 --2- 172.21.15.99:0/3065689926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eb41041a0 0x7f5eb419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 >> 172.21.15.99:0/3065689926 conn(0x7f5eb407a6a0 msgr2=0x7f5eb41017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 shutdown_connections 2024-05-14T17:02:12.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:12.490+0000 7f5ebc278700 1 -- 172.21.15.99:0/3065689926 wait complete. 2024-05-14T17:02:13.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:13 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3065689926' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T17:02:13.061 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-14T17:02:13.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:02:13.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:13 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3065689926' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T17:02:13.343 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:13.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:13 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3065689926' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T17:02:14.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:14 smithi193 ceph-mon[35962]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T17:02:14.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:14 smithi150 ceph-mon[36204]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T17:02:14.475 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:14 smithi099 ceph-mon[31851]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T17:02:14.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 -- 172.21.15.99:0/781507304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb968108490 msgr2=0x7fb96806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:14.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 --2- 172.21.15.99:0/781507304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb968108490 0x7fb96806bcb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb964009990 tx=0x7fb964009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 -- 172.21.15.99:0/781507304 shutdown_connections 2024-05-14T17:02:14.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 --2- 172.21.15.99:0/781507304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb968105fb0 0x7fb968106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 --2- 172.21.15.99:0/781507304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb968108490 0x7fb96806bcb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 --2- 172.21.15.99:0/781507304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb96806c1f0 0x7fb96806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 -- 172.21.15.99:0/781507304 >> 172.21.15.99:0/781507304 conn(0x7fb9680784b0 msgr2=0x7fb9680788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:15.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 -- 172.21.15.99:0/781507304 shutdown_connections 2024-05-14T17:02:15.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 -- 172.21.15.99:0/781507304 wait complete. 2024-05-14T17:02:15.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.973+0000 7fb96f796700 1 Processor -- start 2024-05-14T17:02:15.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 -- start start 2024-05-14T17:02:15.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 0x7fb9681a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb968105fb0 0x7fb9681a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb968108490 0x7fb9681a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb968119d10 con 0x7fb96806c1f0 2024-05-14T17:02:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb968119b90 con 0x7fb968105fb0 2024-05-14T17:02:15.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96f796700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb968119e90 con 0x7fb968108490 2024-05-14T17:02:15.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 0x7fb9681a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 0x7fb9681a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35326/0 (socket says 172.21.15.99:35326) 2024-05-14T17:02:15.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 -- 172.21.15.99:0/811756871 learned_addr learned my addr 172.21.15.99:0/811756871 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:15.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96cd31700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb968105fb0 0x7fb9681a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96dd33700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb968108490 0x7fb9681a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 -- 172.21.15.99:0/811756871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb968108490 msgr2=0x7fb9681a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb968108490 0x7fb9681a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 -- 172.21.15.99:0/811756871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb968105fb0 msgr2=0x7fb9681a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb968105fb0 0x7fb9681a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 -- 172.21.15.99:0/811756871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9681a5210 con 0x7fb96806c1f0 2024-05-14T17:02:15.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.974+0000 7fb96d532700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 0x7fb9681a0200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb9640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:15.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.975+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964027070 con 0x7fb96806c1f0 2024-05-14T17:02:15.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.975+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb964003f20 con 0x7fb96806c1f0 2024-05-14T17:02:15.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.975+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9681a54a0 con 0x7fb96806c1f0 2024-05-14T17:02:15.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.975+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964022ca0 con 0x7fb96806c1f0 2024-05-14T17:02:15.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.975+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9681acd80 con 0x7fb96806c1f0 2024-05-14T17:02:15.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.976+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb968102a90 con 0x7fb96806c1f0 2024-05-14T17:02:15.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.977+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb96402a070 con 0x7fb96806c1f0 2024-05-14T17:02:15.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.978+0000 7fb95e7fc700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb9540b2880 0x7fb9540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:15.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.978+0000 7fb96cd31700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb9540b2880 0x7fb9540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.978+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fb9640e3740 con 0x7fb96806c1f0 2024-05-14T17:02:15.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.978+0000 7fb96cd31700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb9540b2880 0x7fb9540b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb968078c60 tx=0x7fb95800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:15.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:14.981+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb964031070 con 0x7fb96806c1f0 2024-05-14T17:02:15.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.236+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb96806a230 con 0x7fb96806c1f0 2024-05-14T17:02:15.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.237+0000 7fb95e7fc700 1 -- 172.21.15.99:0/811756871 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7fb964013070 con 0x7fb96806c1f0 2024-05-14T17:02:15.447 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:00:30.962944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:02:15.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb9540b2880 msgr2=0x7fb9540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb9540b2880 0x7fb9540b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb968078c60 tx=0x7fb95800b410 comp rx=0 tx=0).stop 2024-05-14T17:02:15.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 msgr2=0x7fb9681a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 0x7fb9681a0200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb9640047b0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 shutdown_connections 2024-05-14T17:02:15.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fb9540b2880 0x7fb9540b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb968108490 0x7fb9681a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb96806c1f0 0x7fb9681a0200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 --2- 172.21.15.99:0/811756871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb968105fb0 0x7fb9681a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 >> 172.21.15.99:0/811756871 conn(0x7fb9680784b0 msgr2=0x7fb968102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:15.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 shutdown_connections 2024-05-14T17:02:15.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:15.241+0000 7fb96f796700 1 -- 172.21.15.99:0/811756871 wait complete. 2024-05-14T17:02:15.621 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:02:15.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:02:16.147 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:16.263 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:16 smithi099 ceph-mon[31851]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:16.270 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:16 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/811756871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:16.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:16 smithi193 ceph-mon[35962]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:16.348 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:16 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/811756871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:16.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:16 smithi150 ceph-mon[36204]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T17:02:16.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:16 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/811756871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:17.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 -- 172.21.15.99:0/1994230147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 msgr2=0x7fbc1c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:17.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 --2- 172.21.15.99:0/1994230147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c10f2e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009990 tx=0x7fbc14009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:17.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 -- 172.21.15.99:0/1994230147 shutdown_connections 2024-05-14T17:02:17.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 --2- 172.21.15.99:0/1994230147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c10f2e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 --2- 172.21.15.99:0/1994230147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc1c108510 0x7fbc1c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 --2- 172.21.15.99:0/1994230147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc1c102430 0x7fbc1c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:17.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 -- 172.21.15.99:0/1994230147 >> 172.21.15.99:0/1994230147 conn(0x7fbc1c0793b0 msgr2=0x7fbc1c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:17.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 -- 172.21.15.99:0/1994230147 shutdown_connections 2024-05-14T17:02:17.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.773+0000 7fbc22138700 1 -- 172.21.15.99:0/1994230147 wait complete. 2024-05-14T17:02:17.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 Processor -- start 2024-05-14T17:02:17.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 -- start start 2024-05-14T17:02:17.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc1c102430 0x7fbc1c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:17.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:17.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc1c108510 0x7fbc1c1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:17.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1126c0 con 0x7fbc1c102df0 2024-05-14T17:02:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c112540 con 0x7fbc1c102430 2024-05-14T17:02:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.774+0000 7fbc22138700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c112840 con 0x7fbc1c108510 2024-05-14T17:02:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35334/0 (socket says 172.21.15.99:35334) 2024-05-14T17:02:18.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 -- 172.21.15.99:0/427199472 learned_addr learned my addr 172.21.15.99:0/427199472 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:18.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1bfff700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc1c108510 0x7fbc1c1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:18.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1b7fe700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc1c102430 0x7fbc1c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:18.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 -- 172.21.15.99:0/427199472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc1c108510 msgr2=0x7fbc1c1a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:18.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc1c108510 0x7fbc1c1a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 -- 172.21.15.99:0/427199472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc1c102430 msgr2=0x7fbc1c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:18.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc1c102430 0x7fbc1c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 -- 172.21.15.99:0/427199472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c1a60c0 con 0x7fbc1c102df0 2024-05-14T17:02:18.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc1affd700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c19b040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b530 tx=0x7fbc1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:18.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10015020 con 0x7fbc1c102df0 2024-05-14T17:02:18.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c1a62f0 con 0x7fbc1c102df0 2024-05-14T17:02:18.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc10011ce0 con 0x7fbc1c102df0 2024-05-14T17:02:18.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10010e40 con 0x7fbc1c102df0 2024-05-14T17:02:18.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.775+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c1a6830 con 0x7fbc1c102df0 2024-05-14T17:02:18.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.776+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c10a9a0 con 0x7fbc1c102df0 2024-05-14T17:02:18.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.778+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbc10018020 con 0x7fbc1c102df0 2024-05-14T17:02:18.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.778+0000 7fbc18ff9700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbc040b2900 0x7fbc040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:18.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.778+0000 7fbc1b7fe700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbc040b2900 0x7fbc040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:18.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.779+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fbc100e1fe0 con 0x7fbc1c102df0 2024-05-14T17:02:18.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.779+0000 7fbc1b7fe700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbc040b2900 0x7fbc040b4dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009f20 tx=0x7fbc0c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:18.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:17.782+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc1001f020 con 0x7fbc1c102df0 2024-05-14T17:02:18.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.037+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc1c06a230 con 0x7fbc1c102df0 2024-05-14T17:02:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.038+0000 7fbc18ff9700 1 -- 172.21.15.99:0/427199472 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7fbc100a9d10 con 0x7fbc1c102df0 2024-05-14T17:02:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:18.366 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:00:30.962944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:02:18.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbc040b2900 msgr2=0x7fbc040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:18.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbc040b2900 0x7fbc040b4dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009f20 tx=0x7fbc0c0094f0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 msgr2=0x7fbc1c19b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:18.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c19b040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b530 tx=0x7fbc1000b840 comp rx=0 tx=0).stop 2024-05-14T17:02:18.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 shutdown_connections 2024-05-14T17:02:18.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fbc040b2900 0x7fbc040b4dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc1c108510 0x7fbc1c1a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc1c102430 0x7fbc1c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 --2- 172.21.15.99:0/427199472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc1c102df0 0x7fbc1c19b040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 >> 172.21.15.99:0/427199472 conn(0x7fbc1c0793b0 msgr2=0x7fbc1c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:18.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 shutdown_connections 2024-05-14T17:02:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:18.040+0000 7fbc22138700 1 -- 172.21.15.99:0/427199472 wait complete. 2024-05-14T17:02:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:02:18.542 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:18 smithi099 ceph-mon[31851]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:18.551 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:18 smithi150 ceph-mon[36204]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:18.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:18 smithi193 ceph-mon[35962]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:18.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:02:18.975 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/427199472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:19.337 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/427199472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/427199472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:20.086 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:20 smithi099 ceph-mon[31851]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:20 smithi193 ceph-mon[35962]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:20.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:20 smithi150 ceph-mon[36204]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:20.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 -- 172.21.15.99:0/3158305166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102df0 msgr2=0x7f5d3010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 --2- 172.21.15.99:0/3158305166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102df0 0x7f5d3010f2e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b0a0 tx=0x7f5d2800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 -- 172.21.15.99:0/3158305166 shutdown_connections 2024-05-14T17:02:20.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 --2- 172.21.15.99:0/3158305166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102df0 0x7f5d3010f2e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 --2- 172.21.15.99:0/3158305166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d30108510 0x7f5d301088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 --2- 172.21.15.99:0/3158305166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102430 0x7f5d301028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 -- 172.21.15.99:0/3158305166 >> 172.21.15.99:0/3158305166 conn(0x7f5d300793b0 msgr2=0x7f5d300797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:20.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 -- 172.21.15.99:0/3158305166 shutdown_connections 2024-05-14T17:02:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.639+0000 7f5d3486c700 1 -- 172.21.15.99:0/3158305166 wait complete. 2024-05-14T17:02:20.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 Processor -- start 2024-05-14T17:02:20.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 -- start start 2024-05-14T17:02:20.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 0x7f5d3019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102df0 0x7f5d3019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:20.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d30108510 0x7f5d30199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:20.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30112660 con 0x7f5d30102430 2024-05-14T17:02:20.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301124e0 con 0x7f5d30108510 2024-05-14T17:02:20.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d3486c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301127e0 con 0x7f5d30102df0 2024-05-14T17:02:20.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 0x7f5d3019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:20.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 0x7f5d3019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35352/0 (socket says 172.21.15.99:35352) 2024-05-14T17:02:20.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 -- 172.21.15.99:0/2746313630 learned_addr learned my addr 172.21.15.99:0/2746313630 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:20.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2dd9b700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102df0 0x7f5d3019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2ed9d700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d30108510 0x7f5d30199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:20.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 -- 172.21.15.99:0/2746313630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102df0 msgr2=0x7f5d3019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:20.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102df0 0x7f5d3019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 -- 172.21.15.99:0/2746313630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d30108510 msgr2=0x7f5d30199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:20.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d30108510 0x7f5d30199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:20.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 -- 172.21.15.99:0/2746313630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3019a1a0 con 0x7f5d30102430 2024-05-14T17:02:20.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2dd9b700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102df0 0x7f5d3019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:20.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.640+0000 7f5d2e59c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 0x7f5d3019f830 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b530 tx=0x7f5d2000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:20.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.641+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20015020 con 0x7f5d30102430 2024-05-14T17:02:21.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.641+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d20011ce0 con 0x7f5d30102430 2024-05-14T17:02:21.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.641+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3019a490 con 0x7f5d30102430 2024-05-14T17:02:21.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.641+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20010d20 con 0x7f5d30102430 2024-05-14T17:02:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.641+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301a6700 con 0x7f5d30102430 2024-05-14T17:02:21.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.642+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3006a230 con 0x7f5d30102430 2024-05-14T17:02:21.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.643+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d20018020 con 0x7f5d30102430 2024-05-14T17:02:21.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.644+0000 7f5d1f7fe700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5d180b2880 0x7f5d180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:21.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.644+0000 7f5d2dd9b700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5d180b2880 0x7f5d180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.644+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f5d200e1ff0 con 0x7f5d30102430 2024-05-14T17:02:21.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.644+0000 7f5d2dd9b700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5d180b2880 0x7f5d180b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d301021a0 tx=0x7f5d2400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:21.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.648+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d2001f020 con 0x7f5d30102430 2024-05-14T17:02:21.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.853+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d301a6990 con 0x7f5d30102430 2024-05-14T17:02:21.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.854+0000 7f5d1f7fe700 1 -- 172.21.15.99:0/2746313630 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22695 (secure 0 0 0) 0x7f5d200a9ce0 con 0x7f5d30102430 2024-05-14T17:02:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:21.215 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:01:04.227948+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:02:21.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5d180b2880 msgr2=0x7f5d180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:21.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5d180b2880 0x7f5d180b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d301021a0 tx=0x7f5d2400b2c0 comp rx=0 tx=0).stop 2024-05-14T17:02:21.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 msgr2=0x7f5d3019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 0x7f5d3019f830 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b530 tx=0x7f5d2000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:02:21.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 shutdown_connections 2024-05-14T17:02:21.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5d180b2880 0x7f5d180b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d30108510 0x7f5d30199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:21.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30102430 0x7f5d3019f830 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:21.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 --2- 172.21.15.99:0/2746313630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d30102df0 0x7f5d3019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 >> 172.21.15.99:0/2746313630 conn(0x7f5d300793b0 msgr2=0x7f5d30106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:21.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 shutdown_connections 2024-05-14T17:02:21.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:20.857+0000 7f5d3486c700 1 -- 172.21.15.99:0/2746313630 wait complete. 2024-05-14T17:02:21.394 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:21 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2746313630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:02:21.450 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:21 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2746313630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:02:21.458 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:21 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2746313630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:02:21.477 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:02:21.726 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:22.162 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:22 smithi099 ceph-mon[31851]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:22.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:22 smithi150 ceph-mon[36204]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:22.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:22 smithi193 ceph-mon[35962]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:23.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 -- 172.21.15.99:0/479306944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 msgr2=0x7f61801028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 --2- 172.21.15.99:0/479306944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f61801028c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6174009990 tx=0x7f6174009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 -- 172.21.15.99:0/479306944 shutdown_connections 2024-05-14T17:02:23.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 --2- 172.21.15.99:0/479306944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6180102e00 0x7f618010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 --2- 172.21.15.99:0/479306944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6180108520 0x7f6180108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 --2- 172.21.15.99:0/479306944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f61801028c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 -- 172.21.15.99:0/479306944 >> 172.21.15.99:0/479306944 conn(0x7f61800793f0 msgr2=0x7f6180079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:23.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 -- 172.21.15.99:0/479306944 shutdown_connections 2024-05-14T17:02:23.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.312+0000 7f61876b4700 1 -- 172.21.15.99:0/479306944 wait complete. 2024-05-14T17:02:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 Processor -- start 2024-05-14T17:02:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 -- start start 2024-05-14T17:02:23.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f6180104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6180102e00 0x7f61801045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6180108520 0x7f6180104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6180112650 con 0x7f6180102440 2024-05-14T17:02:23.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61801124d0 con 0x7f6180102e00 2024-05-14T17:02:23.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f61876b4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61801127d0 con 0x7f6180108520 2024-05-14T17:02:23.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f6185450700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f6180104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f6185450700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f6180104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35366/0 (socket says 172.21.15.99:35366) 2024-05-14T17:02:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f6185450700 1 -- 172.21.15.99:0/2305130722 learned_addr learned my addr 172.21.15.99:0/2305130722 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:23.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f6184c4f700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6180102e00 0x7f61801045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.313+0000 7f6185c51700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6180108520 0x7f6180104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f6185450700 1 -- 172.21.15.99:0/2305130722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6180108520 msgr2=0x7f6180104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:23.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f6185450700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6180108520 0x7f6180104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f6185450700 1 -- 172.21.15.99:0/2305130722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6180102e00 msgr2=0x7f61801045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:23.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f6185450700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6180102e00 0x7f61801045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f6185450700 1 -- 172.21.15.99:0/2305130722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61801a81a0 con 0x7f6180102440 2024-05-14T17:02:23.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f6185450700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f6180104080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f617800d950 tx=0x7f617800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178016070 con 0x7f6180102440 2024-05-14T17:02:23.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6178009df0 con 0x7f6180102440 2024-05-14T17:02:23.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617800f3c0 con 0x7f6180102440 2024-05-14T17:02:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61801a8490 con 0x7f6180102440 2024-05-14T17:02:23.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.314+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61801a89a0 con 0x7f6180102440 2024-05-14T17:02:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.315+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618006a230 con 0x7f6180102440 2024-05-14T17:02:23.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.316+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6178019070 con 0x7f6180102440 2024-05-14T17:02:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.317+0000 7f617e7fc700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f61680b2880 0x7f61680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.317+0000 7f6184c4f700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f61680b2880 0x7f61680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.317+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f61780e1e10 con 0x7f6180102440 2024-05-14T17:02:23.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.318+0000 7f6184c4f700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f61680b2880 0x7f61680b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f617400b5c0 tx=0x7f6174005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.321+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61780a9b20 con 0x7f6180102440 2024-05-14T17:02:23.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.578+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f618002e560 con 0x7f6180102440 2024-05-14T17:02:23.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.578+0000 7f617e7fc700 1 -- 172.21.15.99:0/2305130722 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7f61780a9420 con 0x7f6180102440 2024-05-14T17:02:23.864 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:23.871 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:00:30.962944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:02:23.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f61680b2880 msgr2=0x7f61680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:23.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f61680b2880 0x7f61680b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f617400b5c0 tx=0x7f6174005960 comp rx=0 tx=0).stop 2024-05-14T17:02:23.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 msgr2=0x7f6180104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f6180104080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f617800d950 tx=0x7f617800dc60 comp rx=0 tx=0).stop 2024-05-14T17:02:23.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 shutdown_connections 2024-05-14T17:02:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f61680b2880 0x7f61680b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6180108520 0x7f6180104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6180102440 0x7f6180104080 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 --2- 172.21.15.99:0/2305130722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6180102e00 0x7f61801045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 >> 172.21.15.99:0/2305130722 conn(0x7f61800793f0 msgr2=0x7f618006b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:24.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 shutdown_connections 2024-05-14T17:02:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:23.581+0000 7f61876b4700 1 -- 172.21.15.99:0/2305130722 wait complete. 2024-05-14T17:02:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:02:24.166 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:24 smithi099 ceph-mon[31851]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:24.181 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:24 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2305130722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:24.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T17:00:11.851265+0000', 'modified': '2024-05-14T17:00:30.962944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24463, 'mds_1': 24448, 'mds_2': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/1891176204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 1891176204}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1891176204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3825469703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3825469703}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3825469703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/3113222034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 3113222034}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 3113222034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.150:6839/3105942045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 3105942045}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 3105942045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.193:6837/4135945134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 4135945134}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 4135945134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/4034311425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 4034311425}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 4034311425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24463, 'qdb_cluster': [24448, 24463, 24443]} 2024-05-14T17:02:24.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T17:02:24.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T17:02:24.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:24 smithi193 ceph-mon[35962]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:24.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:24 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2305130722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:24 smithi150 ceph-mon[36204]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:24.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:24 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2305130722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:26.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 -- 172.21.15.99:0/68722652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d8108620 msgr2=0x7f68d8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 --2- 172.21.15.99:0/68722652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d8108620 0x7f68d8108a00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009990 tx=0x7f68d4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 -- 172.21.15.99:0/68722652 shutdown_connections 2024-05-14T17:02:26.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 --2- 172.21.15.99:0/68722652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d81028c0 0x7f68d8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 --2- 172.21.15.99:0/68722652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d8108620 0x7f68d8108a00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 --2- 172.21.15.99:0/68722652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d8101f00 0x7f68d8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 -- 172.21.15.99:0/68722652 >> 172.21.15.99:0/68722652 conn(0x7f68d8077780 msgr2=0x7f68d8077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 -- 172.21.15.99:0/68722652 shutdown_connections 2024-05-14T17:02:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.032+0000 7f68e0444700 1 -- 172.21.15.99:0/68722652 wait complete. 2024-05-14T17:02:26.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 Processor -- start 2024-05-14T17:02:26.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 -- start start 2024-05-14T17:02:26.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d8101f00 0x7f68d819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 0x7f68d819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d8108620 0x7f68d8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8113770 con 0x7f68d81028c0 2024-05-14T17:02:26.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81135f0 con 0x7f68d8101f00 2024-05-14T17:02:26.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68e0444700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81138f0 con 0x7f68d8108620 2024-05-14T17:02:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 0x7f68d819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 0x7f68d819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50638/0 (socket says 172.21.15.99:50638) 2024-05-14T17:02:26.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 -- 172.21.15.99:0/3974888777 learned_addr learned my addr 172.21.15.99:0/3974888777 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:26.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68de1e0700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d8101f00 0x7f68d819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 -- 172.21.15.99:0/3974888777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d8108620 msgr2=0x7f68d8199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d8108620 0x7f68d8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 -- 172.21.15.99:0/3974888777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d8101f00 msgr2=0x7f68d819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d8101f00 0x7f68d819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.033+0000 7f68dd9df700 1 -- 172.21.15.99:0/3974888777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d819a270 con 0x7f68d81028c0 2024-05-14T17:02:26.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.034+0000 7f68dd9df700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 0x7f68d819fe40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c970 tx=0x7f68c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.034+0000 7f68cf7fe700 1 -- 172.21.15.99:0/3974888777 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8007aa0 con 0x7f68d81028c0 2024-05-14T17:02:26.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.034+0000 7f68cf7fe700 1 -- 172.21.15.99:0/3974888777 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c8004ce0 con 0x7f68d81028c0 2024-05-14T17:02:26.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.034+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d819a560 con 0x7f68d81028c0 2024-05-14T17:02:26.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.034+0000 7f68cf7fe700 1 -- 172.21.15.99:0/3974888777 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8005510 con 0x7f68d81028c0 2024-05-14T17:02:26.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.034+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81a67d0 con 0x7f68d81028c0 2024-05-14T17:02:26.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.036+0000 7f68cf7fe700 1 -- 172.21.15.99:0/3974888777 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f68c8007c40 con 0x7f68d81028c0 2024-05-14T17:02:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.037+0000 7f68cf7fe700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f68c40b2900 0x7f68c40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.037+0000 7f68de1e0700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f68c40b2900 0x7f68c40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.037+0000 7f68cf7fe700 1 -- 172.21.15.99:0/3974888777 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f68c80e1f40 con 0x7f68d81028c0 2024-05-14T17:02:26.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.037+0000 7f68de1e0700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f68c40b2900 0x7f68c40b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68d4006010 tx=0x7f68d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 Processor -- start 2024-05-14T17:02:26.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 -- start start 2024-05-14T17:02:26.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ade380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ade8c0 0x5592e9ad87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592e9a36d20 con 0x5592e9ad8dc0 2024-05-14T17:02:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e9a268c0 con 0x5592e9ade8c0 2024-05-14T17:02:26.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68eacdab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592e9a2a900 con 0x5592e9addfa0 2024-05-14T17:02:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50652/0 (socket says 172.21.15.99:50652) 2024-05-14T17:02:26.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 -- 172.21.15.99:0/2203983047 learned_addr learned my addr 172.21.15.99:0/2203983047 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:26.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68dd9df700 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ade8c0 0x5592e9ad87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de1e0700 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ade380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 -- 172.21.15.99:0/2203983047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 msgr2=0x5592e9ade380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:26.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ade380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 -- 172.21.15.99:0/2203983047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ade8c0 msgr2=0x5592e9ad87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:26.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ade8c0 0x5592e9ad87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.038+0000 7f68de9e1700 1 -- 172.21.15.99:0/2203983047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e99cff60 con 0x5592e9ad8dc0 2024-05-14T17:02:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.039+0000 7f68de9e1700 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad9220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009be0 tx=0x7f68d0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1ca581a8fc7dca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.039+0000 7f68ccd70700 1 -- 172.21.15.99:0/2203983047 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004bb0 con 0x5592e9ad8dc0 2024-05-14T17:02:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.039+0000 7f68ccd70700 1 -- 172.21.15.99:0/2203983047 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0004d50 con 0x5592e9ad8dc0 2024-05-14T17:02:26.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.039+0000 7f68ccd70700 1 -- 172.21.15.99:0/2203983047 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00136e0 con 0x5592e9ad8dc0 2024-05-14T17:02:26.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 -- 172.21.15.99:0/2203983047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 msgr2=0x5592e9ad9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad9220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009be0 tx=0x7f68d0009ef0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 -- 172.21.15.99:0/2203983047 shutdown_connections 2024-05-14T17:02:26.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad9220 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ade380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 --2- 172.21.15.99:0/2203983047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ade8c0 0x5592e9ad87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 -- 172.21.15.99:0/2203983047 >> 172.21.15.99:0/2203983047 conn(0x5592e9a05d60 msgr2=0x5592e9ad2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:26.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 -- 172.21.15.99:0/2203983047 shutdown_connections 2024-05-14T17:02:26.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 -- 172.21.15.99:0/2203983047 wait complete. 2024-05-14T17:02:26.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 -- start start 2024-05-14T17:02:26.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.040+0000 7f68eacdab80 1 Processor -- start 2024-05-14T17:02:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ae4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ae18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 0x5592e9ae1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592e992cf50 con 0x5592e9ade8c0 2024-05-14T17:02:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e998ed50 con 0x5592e9ad8dc0 2024-05-14T17:02:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592e9ae52d0 con 0x5592e9addfa0 2024-05-14T17:02:26.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 0x5592e9ae1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 10 client.0 _ll_register_callbacks cb 0x5592e99c1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:26.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 0x5592e9ae1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50658/0 (socket says 172.21.15.99:50658) 2024-05-14T17:02:26.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 -- 172.21.15.99:0/1406909743 learned_addr learned my addr 172.21.15.99:0/1406909743 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:26.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68dd9df700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ae18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de1e0700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ae4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 -- 172.21.15.99:0/1406909743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 msgr2=0x5592e9ae18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:26.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ae18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 -- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 msgr2=0x5592e9ae4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:26.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ae4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 -- 172.21.15.99:0/1406909743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e9976880 con 0x5592e9ade8c0 2024-05-14T17:02:26.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68de9e1700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 0x5592e9ae1e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f68d0015040 tx=0x7f68d0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111ef058c7ef57f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:02:26.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68c1ffb700 1 -- 172.21.15.99:0/1406909743 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0021070 con 0x5592e9ade8c0 2024-05-14T17:02:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d0004d50 con 0x5592e9ade8c0 2024-05-14T17:02:26.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.041+0000 7f68c1ffb700 1 -- 172.21.15.99:0/1406909743 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f68d0005430 con 0x5592e9ade8c0 2024-05-14T17:02:27.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68c1ffb700 1 -- 172.21.15.99:0/1406909743 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68d002d050 con 0x5592e9ade8c0 2024-05-14T17:02:27.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 10 client.14964 fetch_fsmap learned FSMap version 25 2024-05-14T17:02:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592e99e0710 con 0x5592e9ade8c0 2024-05-14T17:02:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68c1ffb700 1 -- 172.21.15.99:0/1406909743 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0004d50 con 0x5592e9ade8c0 2024-05-14T17:02:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68c1ffb700 1 -- 172.21.15.99:0/1406909743 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f68d0012070 con 0x5592e9ade8c0 2024-05-14T17:02:27.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 10 client.14964 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:02:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 10 client.14964 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T17:02:27.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 20 client.14964 populate_metadata read hostname 'smithi099' 2024-05-14T17:02:27.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9af5e20 0x5592e9af7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 4 client.14964 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:27.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 --> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e9a38c70 con 0x5592e9af5e20 2024-05-14T17:02:27.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.042+0000 7f68de1e0700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9af5e20 0x5592e9af7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.043+0000 7f68c1ffb700 10 client.14964 ms_handle_connect on v2:172.21.15.150:6838/3105942045 2024-05-14T17:02:27.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.043+0000 7f68de1e0700 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9af5e20 0x5592e9af7290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef767368d54fdf1e server_cookie=3e13865fef68bab9 in_seq=0 out_seq=0 2024-05-14T17:02:27.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68c1ffb700 1 -- 172.21.15.99:0/1406909743 <== mds.2 v2:172.21.15.150:6838/3105942045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592e9a38c70 con 0x5592e9af5e20 2024-05-14T17:02:27.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68c1ffb700 10 client.14964 handle_command_reply: tid=0 2024-05-14T17:02:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 client.14964 shutdown 2024-05-14T17:02:27.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 msgr2=0x5592e9ae1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 0x5592e9ae1e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f68d0015040 tx=0x7f68d0004f80 comp rx=0 tx=0).stop 2024-05-14T17:02:27.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 shutdown_connections 2024-05-14T17:02:27.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9af5e20 0x5592e9af7290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9ade8c0 0x5592e9ae1e10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ae4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68c1ffb700 0 client.14964 ms_handle_reset on v2:172.21.15.150:6838/3105942045 2024-05-14T17:02:27.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 --2- 172.21.15.99:0/1406909743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9addfa0 0x5592e9ae18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 >> 172.21.15.99:0/1406909743 conn(0x5592e9a05d60 msgr2=0x5592e9ad2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:27.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 shutdown_connections 2024-05-14T17:02:27.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 1 -- 172.21.15.99:0/1406909743 wait complete. 2024-05-14T17:02:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.044+0000 7f68eacdab80 20 client.14964 trim_cache size 0 max 16384 2024-05-14T17:02:27.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 Processor -- start 2024-05-14T17:02:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 -- start start 2024-05-14T17:02:27.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9ae1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9ae3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592e9af55d0 con 0x5592e9addfa0 2024-05-14T17:02:27.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e9af5ca0 con 0x5592e9ad8dc0 2024-05-14T17:02:27.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68eacdab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592e9af6370 con 0x5592e9ade8c0 2024-05-14T17:02:27.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68de9e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9ae3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68dd9df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9ae1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68de1e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68de9e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9ae3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:56916/0 (socket says 172.21.15.99:56916) 2024-05-14T17:02:27.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.068+0000 7f68de9e1700 1 -- 172.21.15.99:0/317045366 learned_addr learned my addr 172.21.15.99:0/317045366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:27.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68de9e1700 1 -- 172.21.15.99:0/317045366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 msgr2=0x5592e9ad87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68de9e1700 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68de9e1700 1 -- 172.21.15.99:0/317045366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 msgr2=0x5592e9ae1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68de9e1700 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9ae1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68de9e1700 1 -- 172.21.15.99:0/317045366 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e9ade6c0 con 0x5592e9ade8c0 2024-05-14T17:02:27.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68dd9df700 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9ae1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:27.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68de9e1700 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9ae3d70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009890 tx=0x7f68d001d6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b0ac312b319363 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:27.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68c3fff700 1 -- 172.21.15.99:0/317045366 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0021070 con 0x5592e9ade8c0 2024-05-14T17:02:27.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68c3fff700 1 -- 172.21.15.99:0/317045366 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0004bb0 con 0x5592e9ade8c0 2024-05-14T17:02:27.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.069+0000 7f68c3fff700 1 -- 172.21.15.99:0/317045366 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d001d710 con 0x5592e9ade8c0 2024-05-14T17:02:27.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 -- 172.21.15.99:0/317045366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 msgr2=0x5592e9ae3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9ae3d70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009890 tx=0x7f68d001d6a0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 -- 172.21.15.99:0/317045366 shutdown_connections 2024-05-14T17:02:27.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9ae3d70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9ad87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 --2- 172.21.15.99:0/317045366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9ae1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 -- 172.21.15.99:0/317045366 >> 172.21.15.99:0/317045366 conn(0x5592e9a05d60 msgr2=0x5592e9ae16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 -- 172.21.15.99:0/317045366 shutdown_connections 2024-05-14T17:02:27.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 -- 172.21.15.99:0/317045366 wait complete. 2024-05-14T17:02:27.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 -- start start 2024-05-14T17:02:27.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.070+0000 7f68eacdab80 1 Processor -- start 2024-05-14T17:02:27.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9b00710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9b00c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9afb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592e9af7110 con 0x5592e9addfa0 2024-05-14T17:02:27.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5592e9af6a40 con 0x5592e9ad8dc0 2024-05-14T17:02:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592e9af77e0 con 0x5592e9ade8c0 2024-05-14T17:02:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 10 client.0 _ll_register_callbacks cb 0x5592e99c1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:27.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9b00c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9b00c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50672/0 (socket says 172.21.15.99:50672) 2024-05-14T17:02:27.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 -- 172.21.15.99:0/271494201 learned_addr learned my addr 172.21.15.99:0/271494201 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68de1e0700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9b00710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68de9e1700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9afb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 -- 172.21.15.99:0/271494201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 msgr2=0x5592e9afb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9afb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 -- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 msgr2=0x5592e9b00710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9b00710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 -- 172.21.15.99:0/271494201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e9afbc00 con 0x5592e9addfa0 2024-05-14T17:02:27.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68de1e0700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9b00710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:27.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68dd9df700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9b00c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c940 tx=0x7f68c80a5b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99f4acb2b548ceb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:27.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:02:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68c1ffb700 1 -- 172.21.15.99:0/271494201 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c80c3e60 con 0x5592e9addfa0 2024-05-14T17:02:27.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68c1ffb700 1 -- 172.21.15.99:0/271494201 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f68c80b2150 con 0x5592e9addfa0 2024-05-14T17:02:27.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.071+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d0004bb0 con 0x5592e9addfa0 2024-05-14T17:02:27.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68c1ffb700 1 -- 172.21.15.99:0/271494201 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68c80c2020 con 0x5592e9addfa0 2024-05-14T17:02:28.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 10 client.14970 fetch_fsmap learned FSMap version 25 2024-05-14T17:02:28.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592e9b61a50 con 0x5592e9addfa0 2024-05-14T17:02:28.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68c1ffb700 1 -- 172.21.15.99:0/271494201 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0004bb0 con 0x5592e9addfa0 2024-05-14T17:02:28.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68c1ffb700 1 -- 172.21.15.99:0/271494201 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f68c80a8860 con 0x5592e9addfa0 2024-05-14T17:02:28.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 10 client.14970 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:02:28.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 10 client.14970 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T17:02:28.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 20 client.14970 populate_metadata read hostname 'smithi099' 2024-05-14T17:02:28.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9b666c0 0x5592e9b653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 4 client.14970 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:02:28.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 --> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] -- command(tid 0: {"prefix": "status"}) -- 0x5592e9a32450 con 0x5592e9b666c0 2024-05-14T17:02:28.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.072+0000 7f68de1e0700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9b666c0 0x5592e9b653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68de1e0700 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9b666c0 0x5592e9b653d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84fdfc4a69871f59 server_cookie=1b14ccbef8c949a4 in_seq=0 out_seq=0 2024-05-14T17:02:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68c1ffb700 10 client.14970 ms_handle_connect on v2:172.21.15.150:6838/3105942045 2024-05-14T17:02:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68c1ffb700 1 -- 172.21.15.99:0/271494201 <== mds.2 v2:172.21.15.150:6838/3105942045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5592e9a32450 con 0x5592e9b666c0 2024-05-14T17:02:28.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68c1ffb700 10 client.14970 handle_command_reply: tid=0 2024-05-14T17:02:28.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 client.14970 shutdown 2024-05-14T17:02:28.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 msgr2=0x5592e9b00c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:28.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9b00c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c940 tx=0x7f68c80a5b40 comp rx=0 tx=0).stop 2024-05-14T17:02:28.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 shutdown_connections 2024-05-14T17:02:28.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5592e9b666c0 0x5592e9b653d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592e9ade8c0 0x5592e9afb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5592e9ad8dc0 0x5592e9b00710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 --2- 172.21.15.99:0/271494201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592e9addfa0 0x5592e9b00c50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 >> 172.21.15.99:0/271494201 conn(0x5592e9a05d60 msgr2=0x5592e9adeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:28.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68c1ffb700 0 client.14970 ms_handle_reset on v2:172.21.15.150:6838/3105942045 2024-05-14T17:02:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 shutdown_connections 2024-05-14T17:02:28.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 1 -- 172.21.15.99:0/271494201 wait complete. 2024-05-14T17:02:28.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.073+0000 7f68eacdab80 20 client.14970 trim_cache size 0 max 16384 2024-05-14T17:02:28.295 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:02:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:02:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-05-14T17:02:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24443, 2024-05-14T17:02:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:02:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:02:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:02:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 119.95718710200001, 2024-05-14T17:02:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-05-14T17:02:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 82, 2024-05-14T17:02:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T17:02:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 151.53782473699999 2024-05-14T17:02:28.413 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:02:28.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f68c40b2900 msgr2=0x7f68c40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:28.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f68c40b2900 0x7f68c40b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68d4006010 tx=0x7f68d4005960 comp rx=0 tx=0).stop 2024-05-14T17:02:28.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 msgr2=0x7f68d819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:28.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 0x7f68d819fe40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c970 tx=0x7f68c800cc80 comp rx=0 tx=0).stop 2024-05-14T17:02:28.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 shutdown_connections 2024-05-14T17:02:28.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f68c40b2900 0x7f68c40b4dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d8108620 0x7f68d8199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d8101f00 0x7f68d819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 --2- 172.21.15.99:0/3974888777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68d81028c0 0x7f68d819fe40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 >> 172.21.15.99:0/3974888777 conn(0x7f68d8077780 msgr2=0x7f68d8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 shutdown_connections 2024-05-14T17:02:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:26.074+0000 7f68e0444700 1 -- 172.21.15.99:0/3974888777 wait complete. 2024-05-14T17:02:28.595 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:26 smithi099 ceph-mon[31851]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:28.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-14T17:02:28.614 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:28 smithi099 ceph-mon[31851]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:28.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:26 smithi193 ceph-mon[35962]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:28.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:28 smithi193 ceph-mon[35962]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:28.640 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:26 smithi150 ceph-mon[36204]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:28.649 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:28 smithi150 ceph-mon[36204]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:28.924 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:30.251 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:30 smithi099 ceph-mon[31851]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:30.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:30 smithi150 ceph-mon[36204]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 -- 172.21.15.99:0/3926534174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 msgr2=0x7fab24109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:30.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 --2- 172.21.15.99:0/3926534174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24109990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009990 tx=0x7fab1c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 -- 172.21.15.99:0/3926534174 shutdown_connections 2024-05-14T17:02:30.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 --2- 172.21.15.99:0/3926534174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406bbe0 0x7fab241103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 --2- 172.21.15.99:0/3926534174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24109990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 --2- 172.21.15.99:0/3926534174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406b220 0x7fab2406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 -- 172.21.15.99:0/3926534174 >> 172.21.15.99:0/3926534174 conn(0x7fab240793f0 msgr2=0x7fab24079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:30.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 -- 172.21.15.99:0/3926534174 shutdown_connections 2024-05-14T17:02:30.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 -- 172.21.15.99:0/3926534174 wait complete. 2024-05-14T17:02:30.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.478+0000 7fab2c193700 1 Processor -- start 2024-05-14T17:02:30.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 -- start start 2024-05-14T17:02:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406b220 0x7fab24102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:30.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406bbe0 0x7fab24102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:30.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:30.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab241137c0 con 0x7fab241095b0 2024-05-14T17:02:30.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab24113640 con 0x7fab2406b220 2024-05-14T17:02:30.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2c193700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab24113940 con 0x7fab2406bbe0 2024-05-14T17:02:30.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:30.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50688/0 (socket says 172.21.15.99:50688) 2024-05-14T17:02:30.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 -- 172.21.15.99:0/737489088 learned_addr learned my addr 172.21.15.99:0/737489088 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:30.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2972e700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406bbe0 0x7fab24102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:30.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab29f2f700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406b220 0x7fab24102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:30.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 -- 172.21.15.99:0/737489088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406bbe0 msgr2=0x7fab24102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406bbe0 0x7fab24102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 -- 172.21.15.99:0/737489088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406b220 msgr2=0x7fab24102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406b220 0x7fab24102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 -- 172.21.15.99:0/737489088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab241a82a0 con 0x7fab241095b0 2024-05-14T17:02:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2972e700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406bbe0 0x7fab24102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:30.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.479+0000 7fab2a730700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24102f90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b550 tx=0x7fab1400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:30.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.480+0000 7fab22ffd700 1 -- 172.21.15.99:0/737489088 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14015020 con 0x7fab241095b0 2024-05-14T17:02:30.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.480+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241a8590 con 0x7fab241095b0 2024-05-14T17:02:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.480+0000 7fab22ffd700 1 -- 172.21.15.99:0/737489088 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab14011ce0 con 0x7fab241095b0 2024-05-14T17:02:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.480+0000 7fab22ffd700 1 -- 172.21.15.99:0/737489088 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14010d40 con 0x7fab241095b0 2024-05-14T17:02:31.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.480+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241a8a80 con 0x7fab241095b0 2024-05-14T17:02:31.079 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:30 smithi193 ceph-mon[35962]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:02:31.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.482+0000 7fab22ffd700 1 -- 172.21.15.99:0/737489088 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fab14018020 con 0x7fab241095b0 2024-05-14T17:02:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.483+0000 7fab22ffd700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fab0c0b2880 0x7fab0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.483+0000 7fab29f2f700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fab0c0b2880 0x7fab0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.483+0000 7fab22ffd700 1 -- 172.21.15.99:0/737489088 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fab140e19f0 con 0x7fab241095b0 2024-05-14T17:02:31.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.483+0000 7fab36a29b80 1 Processor -- start 2024-05-14T17:02:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.483+0000 7fab36a29b80 1 -- start start 2024-05-14T17:02:31.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.483+0000 7fab29f2f700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fab0c0b2880 0x7fab0c0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab1c006010 tx=0x7fab1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:31.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f082cc10 0x5620f08303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f0830910 0x5620f0830d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab36a29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620f078ac40 con 0x5620f082c2f0 2024-05-14T17:02:31.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab36a29b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620f077a7e0 con 0x5620f082cc10 2024-05-14T17:02:31.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab36a29b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620f077e820 con 0x5620f0830910 2024-05-14T17:02:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50698/0 (socket says 172.21.15.99:50698) 2024-05-14T17:02:31.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 -- 172.21.15.99:0/1333938562 learned_addr learned my addr 172.21.15.99:0/1333938562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:31.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab2972e700 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f082cc10 0x5620f08303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab2a730700 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f0830910 0x5620f0830d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 -- 172.21.15.99:0/1333938562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f0830910 msgr2=0x5620f0830d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:31.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f0830910 0x5620f0830d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 -- 172.21.15.99:0/1333938562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f082cc10 msgr2=0x5620f08303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:31.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f082cc10 0x5620f08303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 -- 172.21.15.99:0/1333938562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f06ca880 con 0x5620f082c2f0 2024-05-14T17:02:31.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab29f2f700 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082c6d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab1c015000 tx=0x7fab1c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3c1d0a563d763c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:31.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab0bfff700 1 -- 172.21.15.99:0/1333938562 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00fa70 con 0x5620f082c2f0 2024-05-14T17:02:31.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab0bfff700 1 -- 172.21.15.99:0/1333938562 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c032070 con 0x5620f082c2f0 2024-05-14T17:02:31.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.484+0000 7fab0bfff700 1 -- 172.21.15.99:0/1333938562 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c023dd0 con 0x5620f082c2f0 2024-05-14T17:02:31.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 -- 172.21.15.99:0/1333938562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 msgr2=0x5620f082c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:31.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082c6d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab1c015000 tx=0x7fab1c01e000 comp rx=0 tx=0).stop 2024-05-14T17:02:31.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 -- 172.21.15.99:0/1333938562 shutdown_connections 2024-05-14T17:02:31.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f0830910 0x5620f0830d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082c6d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 --2- 172.21.15.99:0/1333938562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f082cc10 0x5620f08303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 -- 172.21.15.99:0/1333938562 >> 172.21.15.99:0/1333938562 conn(0x5620f0757200 msgr2=0x5620f0826930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:31.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 -- 172.21.15.99:0/1333938562 shutdown_connections 2024-05-14T17:02:31.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.485+0000 7fab36a29b80 1 -- 172.21.15.99:0/1333938562 wait complete. 2024-05-14T17:02:31.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 -- start start 2024-05-14T17:02:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 Processor -- start 2024-05-14T17:02:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f0837dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f0835160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f08356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620f0680f50 con 0x5620f082cc10 2024-05-14T17:02:31.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620f071c770 con 0x5620f0830910 2024-05-14T17:02:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620f0838b10 con 0x5620f082c2f0 2024-05-14T17:02:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab36a29b80 10 client.0 _ll_register_callbacks cb 0x5620f0711530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:31.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab2972e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f0835160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab2972e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f0835160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50700/0 (socket says 172.21.15.99:50700) 2024-05-14T17:02:31.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.486+0000 7fab2972e700 1 -- 172.21.15.99:0/3471902877 learned_addr learned my addr 172.21.15.99:0/3471902877 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:31.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f0837dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2972e700 1 -- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 msgr2=0x5620f0837dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:31.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2972e700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f0837dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2972e700 1 -- 172.21.15.99:0/3471902877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 msgr2=0x5620f08356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:31.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2a730700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f08356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:31.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2972e700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f08356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:31.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2972e700 1 -- 172.21.15.99:0/3471902877 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f0719640 con 0x5620f082cc10 2024-05-14T17:02:31.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f0837dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab2972e700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f0835160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab1800ce90 tx=0x7fab1800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd2aefd8475b05d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:31.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab097fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:02:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab097fa700 1 -- 172.21.15.99:0/3471902877 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18007920 con 0x5620f082cc10 2024-05-14T17:02:31.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab097fa700 1 -- 172.21.15.99:0/3471902877 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fab18007ac0 con 0x5620f082cc10 2024-05-14T17:02:31.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c032070 con 0x5620f082cc10 2024-05-14T17:02:31.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab097fa700 1 -- 172.21.15.99:0/3471902877 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab18007cc0 con 0x5620f082cc10 2024-05-14T17:02:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab36a29b80 10 client.14982 fetch_fsmap learned FSMap version 25 2024-05-14T17:02:32.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620f0843200 con 0x5620f082cc10 2024-05-14T17:02:32.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab097fa700 1 -- 172.21.15.99:0/3471902877 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c032070 con 0x5620f082cc10 2024-05-14T17:02:32.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab097fa700 1 -- 172.21.15.99:0/3471902877 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fab1801f400 con 0x5620f082cc10 2024-05-14T17:02:32.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.487+0000 7fab36a29b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:02:32.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab36a29b80 10 client.14982 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T17:02:32.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab36a29b80 20 client.14982 populate_metadata read hostname 'smithi099' 2024-05-14T17:02:32.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f0846d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab36a29b80 4 client.14982 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 --> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f07f8310 con 0x5620f08458c0 2024-05-14T17:02:32.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f0846d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f0846d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b2f2ae3fe956f71 server_cookie=d553ac56fb63c175 in_seq=0 out_seq=0 2024-05-14T17:02:32.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.488+0000 7fab097fa700 10 client.14982 ms_handle_connect on v2:172.21.15.193:6836/4135945134 2024-05-14T17:02:32.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.489+0000 7fab097fa700 1 -- 172.21.15.99:0/3471902877 <== mds.1 v2:172.21.15.193:6836/4135945134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620f07f8310 con 0x5620f08458c0 2024-05-14T17:02:32.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.489+0000 7fab097fa700 10 client.14982 handle_command_reply: tid=0 2024-05-14T17:02:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.489+0000 7fab36a29b80 1 client.14982 shutdown 2024-05-14T17:02:32.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.489+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 msgr2=0x5620f0835160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:32.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.489+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f0835160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab1800ce90 tx=0x7fab1800cf70 comp rx=0 tx=0).stop 2024-05-14T17:02:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 shutdown_connections 2024-05-14T17:02:32.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f0846d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f08356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab097fa700 0 client.14982 ms_handle_reset on v2:172.21.15.193:6836/4135945134 2024-05-14T17:02:32.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f0837dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3471902877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f0835160 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 >> 172.21.15.99:0/3471902877 conn(0x5620f0757200 msgr2=0x5620f0831de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:32.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 shutdown_connections 2024-05-14T17:02:32.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 1 -- 172.21.15.99:0/3471902877 wait complete. 2024-05-14T17:02:32.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.490+0000 7fab36a29b80 20 client.14982 trim_cache size 0 max 16384 2024-05-14T17:02:32.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 Processor -- start 2024-05-14T17:02:32.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 -- start start 2024-05-14T17:02:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082cc10 0x5620f082e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620f0845070 con 0x5620f082c2f0 2024-05-14T17:02:32.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620f0845740 con 0x5620f0830910 2024-05-14T17:02:32.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab36a29b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620f0845e10 con 0x5620f082cc10 2024-05-14T17:02:32.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50716/0 (socket says 172.21.15.99:50716) 2024-05-14T17:02:32.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 -- 172.21.15.99:0/3733683473 learned_addr learned my addr 172.21.15.99:0/3733683473 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:32.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab2972e700 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082cc10 0x5620f082e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 -- 172.21.15.99:0/3733683473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082cc10 msgr2=0x5620f082e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:32.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082cc10 0x5620f082e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 -- 172.21.15.99:0/3733683473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 msgr2=0x5620f082e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:32.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 -- 172.21.15.99:0/3733683473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f08336d0 con 0x5620f082c2f0 2024-05-14T17:02:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.511+0000 7fab29f2f700 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082b810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00f740 tx=0x7fab1c019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15296c89d0845540 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:32.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab0b7fe700 1 -- 172.21.15.99:0/3733683473 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c042070 con 0x5620f082c2f0 2024-05-14T17:02:32.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab0b7fe700 1 -- 172.21.15.99:0/3733683473 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c032040 con 0x5620f082c2f0 2024-05-14T17:02:32.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab0b7fe700 1 -- 172.21.15.99:0/3733683473 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c023c20 con 0x5620f082c2f0 2024-05-14T17:02:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 -- 172.21.15.99:0/3733683473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 msgr2=0x5620f082b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:32.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082b810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00f740 tx=0x7fab1c019040 comp rx=0 tx=0).stop 2024-05-14T17:02:32.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 -- 172.21.15.99:0/3733683473 shutdown_connections 2024-05-14T17:02:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082c2f0 0x5620f082b810 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 --2- 172.21.15.99:0/3733683473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082cc10 0x5620f082e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 -- 172.21.15.99:0/3733683473 >> 172.21.15.99:0/3733683473 conn(0x5620f0757200 msgr2=0x5620f082b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:32.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 -- 172.21.15.99:0/3733683473 shutdown_connections 2024-05-14T17:02:32.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.512+0000 7fab36a29b80 1 -- 172.21.15.99:0/3733683473 wait complete. 2024-05-14T17:02:32.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 -- start start 2024-05-14T17:02:32.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 Processor -- start 2024-05-14T17:02:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f082de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f082e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620f0846bb0 con 0x5620f082cc10 2024-05-14T17:02:32.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620f08464e0 con 0x5620f0830910 2024-05-14T17:02:32.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620f0847280 con 0x5620f082c2f0 2024-05-14T17:02:32.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab36a29b80 10 client.0 _ll_register_callbacks cb 0x5620f0711530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:32.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab2972e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f082e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab2972e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f082e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50730/0 (socket says 172.21.15.99:50730) 2024-05-14T17:02:33.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab2972e700 1 -- 172.21.15.99:0/646818613 learned_addr learned my addr 172.21.15.99:0/646818613 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:33.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.513+0000 7fab2a730700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab29f2f700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f082de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab2972e700 1 -- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 msgr2=0x5620f082de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab2972e700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f082de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab2972e700 1 -- 172.21.15.99:0/646818613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 msgr2=0x5620f082e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:33.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab2972e700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab2972e700 1 -- 172.21.15.99:0/646818613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f084e530 con 0x5620f082cc10 2024-05-14T17:02:33.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab2972e700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f082e360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab1801f820 tx=0x7fab1801fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d079497553a9c2cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:33.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab097fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:02:33.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab097fa700 1 -- 172.21.15.99:0/646818613 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18007e60 con 0x5620f082cc10 2024-05-14T17:02:33.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab097fa700 1 -- 172.21.15.99:0/646818613 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fab18007930 con 0x5620f082cc10 2024-05-14T17:02:33.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c032040 con 0x5620f082cc10 2024-05-14T17:02:33.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab097fa700 1 -- 172.21.15.99:0/646818613 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab18022480 con 0x5620f082cc10 2024-05-14T17:02:33.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab36a29b80 10 client.14988 fetch_fsmap learned FSMap version 25 2024-05-14T17:02:33.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.514+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620f08b10b0 con 0x5620f082cc10 2024-05-14T17:02:33.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab097fa700 1 -- 172.21.15.99:0/646818613 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c032040 con 0x5620f082cc10 2024-05-14T17:02:33.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab097fa700 1 -- 172.21.15.99:0/646818613 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fab1802b460 con 0x5620f082cc10 2024-05-14T17:02:33.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab36a29b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:02:33.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab36a29b80 10 client.14988 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T17:02:33.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab36a29b80 20 client.14988 populate_metadata read hostname 'smithi099' 2024-05-14T17:02:33.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab36a29b80 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f08b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab36a29b80 4 client.14988 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:02:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 --> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] -- command(tid 0: {"prefix": "status"}) -- 0x5620f078cb90 con 0x5620f08458c0 2024-05-14T17:02:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.515+0000 7fab29f2f700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f08b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab29f2f700 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f08b4920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=129705a0a78cadc server_cookie=3f69a27d62b0ba2 in_seq=0 out_seq=0 2024-05-14T17:02:33.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab097fa700 10 client.14988 ms_handle_connect on v2:172.21.15.193:6836/4135945134 2024-05-14T17:02:33.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab097fa700 1 -- 172.21.15.99:0/646818613 <== mds.1 v2:172.21.15.193:6836/4135945134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5620f078cb90 con 0x5620f08458c0 2024-05-14T17:02:33.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab097fa700 10 client.14988 handle_command_reply: tid=0 2024-05-14T17:02:33.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 client.14988 shutdown 2024-05-14T17:02:33.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 msgr2=0x5620f082e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f082e360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fab1801f820 tx=0x7fab1801fd40 comp rx=0 tx=0).stop 2024-05-14T17:02:33.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 shutdown_connections 2024-05-14T17:02:33.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5620f08458c0 0x5620f08b4920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5620f0830910 0x5620f082e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f082c2f0 0x5620f082de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab097fa700 0 client.14988 ms_handle_reset on v2:172.21.15.193:6836/4135945134 2024-05-14T17:02:33.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 --2- 172.21.15.99:0/646818613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620f082cc10 0x5620f082e360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 >> 172.21.15.99:0/646818613 conn(0x5620f0757200 msgr2=0x5620f082b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 shutdown_connections 2024-05-14T17:02:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 1 -- 172.21.15.99:0/646818613 wait complete. 2024-05-14T17:02:33.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.516+0000 7fab36a29b80 20 client.14988 trim_cache size 0 max 16384 2024-05-14T17:02:33.642 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:02:33.677 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:02:33.687 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-05-14T17:02:33.701 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24448, 2024-05-14T17:02:33.714 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:02:33.728 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:02:33.740 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:02:33.752 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 125.40705379800001, 2024-05-14T17:02:33.764 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-05-14T17:02:33.773 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 82, 2024-05-14T17:02:33.808 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T17:02:33.820 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 170.94759502599999 2024-05-14T17:02:33.832 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:02:33.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fab0c0b2880 msgr2=0x7fab0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fab0c0b2880 0x7fab0c0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab1c006010 tx=0x7fab1c01a040 comp rx=0 tx=0).stop 2024-05-14T17:02:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 msgr2=0x7fab24102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24102f90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b550 tx=0x7fab1400b910 comp rx=0 tx=0).stop 2024-05-14T17:02:33.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 shutdown_connections 2024-05-14T17:02:33.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fab0c0b2880 0x7fab0c0b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab241095b0 0x7fab24102f90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406b220 0x7fab24102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 --2- 172.21.15.99:0/737489088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab2406bbe0 0x7fab24102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 >> 172.21.15.99:0/737489088 conn(0x7fab240793f0 msgr2=0x7fab24101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:33.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 shutdown_connections 2024-05-14T17:02:33.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:30.517+0000 7fab2c193700 1 -- 172.21.15.99:0/737489088 wait complete. 2024-05-14T17:02:33.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T17:02:34.001 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:32 smithi099 ceph-mon[31851]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:34.029 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:32 smithi193 ceph-mon[35962]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:34.039 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:32 smithi150 ceph-mon[36204]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:34.285 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:34.406 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:34 smithi099 ceph-mon[31851]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:34.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:34 smithi150 ceph-mon[36204]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:34.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:34 smithi193 ceph-mon[35962]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:35.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 -- 172.21.15.99:0/593525504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 msgr2=0x7f3740109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:35.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 --2- 172.21.15.99:0/593525504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740109a10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f373c0099c0 tx=0x7f373c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 -- 172.21.15.99:0/593525504 shutdown_connections 2024-05-14T17:02:35.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 --2- 172.21.15.99:0/593525504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740102470 0x7f3740110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 --2- 172.21.15.99:0/593525504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740109a10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 --2- 172.21.15.99:0/593525504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3740101ab0 0x7f3740101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.876+0000 7f37472f3700 1 -- 172.21.15.99:0/593525504 >> 172.21.15.99:0/593525504 conn(0x7f3740078a40 msgr2=0x7f37400ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:35.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 -- 172.21.15.99:0/593525504 shutdown_connections 2024-05-14T17:02:35.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 -- 172.21.15.99:0/593525504 wait complete. 2024-05-14T17:02:35.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 Processor -- start 2024-05-14T17:02:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 -- start start 2024-05-14T17:02:36.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740101ab0 0x7f374019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3740102470 0x7f374019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3740113790 con 0x7f3740109630 2024-05-14T17:02:36.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3740113610 con 0x7f3740101ab0 2024-05-14T17:02:36.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.877+0000 7f37472f3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3740113910 con 0x7f3740102470 2024-05-14T17:02:36.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54794/0 (socket says 172.21.15.99:54794) 2024-05-14T17:02:36.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 -- 172.21.15.99:0/2415567489 learned_addr learned my addr 172.21.15.99:0/2415567489 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:36.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f374488e700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3740102470 0x7f374019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f374508f700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740101ab0 0x7f374019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 -- 172.21.15.99:0/2415567489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3740102470 msgr2=0x7f374019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3740102470 0x7f374019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 -- 172.21.15.99:0/2415567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740101ab0 msgr2=0x7f374019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:36.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740101ab0 0x7f374019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 -- 172.21.15.99:0/2415567489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374019a090 con 0x7f3740109630 2024-05-14T17:02:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f374508f700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740101ab0 0x7f374019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f3745890700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740199930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f373800cca0 tx=0x7f373800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:36.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f37367fc700 1 -- 172.21.15.99:0/2415567489 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738016020 con 0x7f3740109630 2024-05-14T17:02:36.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f37367fc700 1 -- 172.21.15.99:0/2415567489 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373800aba0 con 0x7f3740109630 2024-05-14T17:02:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374019a380 con 0x7f3740109630 2024-05-14T17:02:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f37367fc700 1 -- 172.21.15.99:0/2415567489 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380115e0 con 0x7f3740109630 2024-05-14T17:02:36.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.878+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401a6640 con 0x7f3740109630 2024-05-14T17:02:36.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.881+0000 7f37367fc700 1 -- 172.21.15.99:0/2415567489 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3738019020 con 0x7f3740109630 2024-05-14T17:02:36.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.881+0000 7f37367fc700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f372c0b2880 0x7f372c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.881+0000 7f374508f700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f372c0b2880 0x7f372c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.881+0000 7f37367fc700 1 -- 172.21.15.99:0/2415567489 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f37380e20e0 con 0x7f3740109630 2024-05-14T17:02:36.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f374508f700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f372c0b2880 0x7f372c0b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f373c00b580 tx=0x7f373c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:36.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 Processor -- start 2024-05-14T17:02:36.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 -- start start 2024-05-14T17:02:36.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18fb28c0 0x555b18fac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 0x555b18fad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b18f0ad20 con 0x555b18facdc0 2024-05-14T17:02:36.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b18efa8c0 con 0x555b18fb1fa0 2024-05-14T17:02:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.882+0000 7f3751b89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b18efe900 con 0x555b18fb28c0 2024-05-14T17:02:36.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 0x555b18fad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 0x555b18fad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54798/0 (socket says 172.21.15.99:54798) 2024-05-14T17:02:36.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 -- 172.21.15.99:0/1229739897 learned_addr learned my addr 172.21.15.99:0/1229739897 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:36.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f374488e700 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18fb28c0 0x555b18fac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f374508f700 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:36.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 -- 172.21.15.99:0/1229739897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18fb28c0 msgr2=0x555b18fac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:36.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18fb28c0 0x555b18fac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 -- 172.21.15.99:0/1229739897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 msgr2=0x555b18fb2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 -- 172.21.15.99:0/1229739897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b18ea3f60 con 0x555b18facdc0 2024-05-14T17:02:36.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f374508f700 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:36.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f3745890700 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 0x555b18fad220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f37380a5790 tx=0x7f37380a8fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e2dac0b587db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:36.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f372b7fe700 1 -- 172.21.15.99:0/1229739897 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380a9220 con 0x555b18facdc0 2024-05-14T17:02:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f372b7fe700 1 -- 172.21.15.99:0/1229739897 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37380a9e10 con 0x555b18facdc0 2024-05-14T17:02:36.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.883+0000 7f372b7fe700 1 -- 172.21.15.99:0/1229739897 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380b20e0 con 0x555b18facdc0 2024-05-14T17:02:36.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 -- 172.21.15.99:0/1229739897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 msgr2=0x555b18fad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:36.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 0x555b18fad220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f37380a5790 tx=0x7f37380a8fb0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 -- 172.21.15.99:0/1229739897 shutdown_connections 2024-05-14T17:02:36.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18facdc0 0x555b18fad220 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 --2- 172.21.15.99:0/1229739897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18fb28c0 0x555b18fac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 -- 172.21.15.99:0/1229739897 >> 172.21.15.99:0/1229739897 conn(0x555b18ed9d60 msgr2=0x555b18fa6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:36.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 -- 172.21.15.99:0/1229739897 shutdown_connections 2024-05-14T17:02:36.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 -- 172.21.15.99:0/1229739897 wait complete. 2024-05-14T17:02:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 -- start start 2024-05-14T17:02:36.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.884+0000 7f3751b89b80 1 Processor -- start 2024-05-14T17:02:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fb8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 0x555b18fb5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:36.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b18e00f50 con 0x555b18fb28c0 2024-05-14T17:02:37.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b18e62d50 con 0x555b18fb1fa0 2024-05-14T17:02:37.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b18fb92d0 con 0x555b18facdc0 2024-05-14T17:02:37.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3751b89b80 10 client.0 _ll_register_callbacks cb 0x555b18e95900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:37.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 0x555b18fb5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 0x555b18fb5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54814/0 (socket says 172.21.15.99:54814) 2024-05-14T17:02:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 -- 172.21.15.99:0/3291488469 learned_addr learned my addr 172.21.15.99:0/3291488469 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f374488e700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f374508f700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fb8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 -- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 msgr2=0x555b18fb8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fb8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 -- 172.21.15.99:0/3291488469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 msgr2=0x555b18fb5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:37.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 -- 172.21.15.99:0/3291488469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b18e4a880 con 0x555b18fb28c0 2024-05-14T17:02:37.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3745890700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 0x555b18fb5f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37380dfc40 tx=0x7f3738091340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47167e4b0f60412 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.885+0000 7f3728ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:02:37.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3728ff9700 1 -- 172.21.15.99:0/3291488469 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380b2570 con 0x555b18fb28c0 2024-05-14T17:02:37.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3728ff9700 1 -- 172.21.15.99:0/3291488469 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f37380b22f0 con 0x555b18fb28c0 2024-05-14T17:02:37.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37380a9e10 con 0x555b18fb28c0 2024-05-14T17:02:37.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3728ff9700 1 -- 172.21.15.99:0/3291488469 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37380a9220 con 0x555b18fb28c0 2024-05-14T17:02:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 10 client.15000 fetch_fsmap learned FSMap version 25 2024-05-14T17:02:37.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b18eb4710 con 0x555b18fb28c0 2024-05-14T17:02:37.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3728ff9700 1 -- 172.21.15.99:0/3291488469 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37380a9e10 con 0x555b18fb28c0 2024-05-14T17:02:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3728ff9700 1 -- 172.21.15.99:0/3291488469 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f37380a83e0 con 0x555b18fb28c0 2024-05-14T17:02:37.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:02:37.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 10 client.15000 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T17:02:37.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 20 client.15000 populate_metadata read hostname 'smithi099' 2024-05-14T17:02:37.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b18fc4bf0 0x555b18fc6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 4 client.15000 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b18f0cc70 con 0x555b18fc4bf0 2024-05-14T17:02:37.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.886+0000 7f374508f700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b18fc4bf0 0x555b18fc6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.887+0000 7f3728ff9700 10 client.15000 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:02:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.887+0000 7f374508f700 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b18fc4bf0 0x555b18fc6d70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da12dcf3b0882ab6 server_cookie=9db7ad4a94c7e601 in_seq=0 out_seq=0 2024-05-14T17:02:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.887+0000 7f3728ff9700 1 -- 172.21.15.99:0/3291488469 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b18f0cc70 con 0x555b18fc4bf0 2024-05-14T17:02:37.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3728ff9700 10 client.15000 handle_command_reply: tid=0 2024-05-14T17:02:37.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 client.15000 shutdown 2024-05-14T17:02:37.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 msgr2=0x555b18fb5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:37.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 0x555b18fb5f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37380dfc40 tx=0x7f3738091340 comp rx=0 tx=0).stop 2024-05-14T17:02:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 shutdown_connections 2024-05-14T17:02:37.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b18fc4bf0 0x555b18fc6d70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb28c0 0x555b18fb5f70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fb8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3728ff9700 0 client.15000 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:02:37.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 --2- 172.21.15.99:0/3291488469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb1fa0 0x555b18fb5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 >> 172.21.15.99:0/3291488469 conn(0x555b18ed9d60 msgr2=0x555b18fa6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:37.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 shutdown_connections 2024-05-14T17:02:37.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 1 -- 172.21.15.99:0/3291488469 wait complete. 2024-05-14T17:02:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.888+0000 7f3751b89b80 20 client.15000 trim_cache size 0 max 16384 2024-05-14T17:02:37.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 Processor -- start 2024-05-14T17:02:37.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 -- start start 2024-05-14T17:02:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fa6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fb33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b18fc50b0 con 0x555b18fb1fa0 2024-05-14T17:02:37.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b18fc5780 con 0x555b18fb28c0 2024-05-14T17:02:37.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f3751b89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b18fc5e50 con 0x555b18facdc0 2024-05-14T17:02:37.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f374488e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fa6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f374488e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fa6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54828/0 (socket says 172.21.15.99:54828) 2024-05-14T17:02:37.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.909+0000 7f374488e700 1 -- 172.21.15.99:0/2598312507 learned_addr learned my addr 172.21.15.99:0/2598312507 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:37.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374508f700 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374488e700 1 -- 172.21.15.99:0/2598312507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 msgr2=0x555b18fac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:37.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374488e700 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fac7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374488e700 1 -- 172.21.15.99:0/2598312507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 msgr2=0x555b18fb33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:37.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374488e700 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fb33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:37.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374488e700 1 -- 172.21.15.99:0/2598312507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b18fb3fa0 con 0x555b18fb1fa0 2024-05-14T17:02:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f374488e700 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fa6db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x555b18fa7a40 tx=0x7f3730009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668b6c7d7cd34635 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:37.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f372affd700 1 -- 172.21.15.99:0/2598312507 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730022070 con 0x555b18fb1fa0 2024-05-14T17:02:37.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f372affd700 1 -- 172.21.15.99:0/2598312507 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3730014c70 con 0x555b18fb1fa0 2024-05-14T17:02:37.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f372affd700 1 -- 172.21.15.99:0/2598312507 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373001dd70 con 0x555b18fb1fa0 2024-05-14T17:02:37.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f3751b89b80 1 -- 172.21.15.99:0/2598312507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 msgr2=0x555b18fa6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:37.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.910+0000 7f3751b89b80 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fa6db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x555b18fa7a40 tx=0x7f3730009b60 comp rx=0 tx=0).stop 2024-05-14T17:02:38.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 -- 172.21.15.99:0/2598312507 shutdown_connections 2024-05-14T17:02:38.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fb33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:38.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 --2- 172.21.15.99:0/2598312507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fa6db0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:38.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 -- 172.21.15.99:0/2598312507 >> 172.21.15.99:0/2598312507 conn(0x555b18ed9d60 msgr2=0x555b18fb2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:38.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 -- 172.21.15.99:0/2598312507 shutdown_connections 2024-05-14T17:02:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 -- 172.21.15.99:0/2598312507 wait complete. 2024-05-14T17:02:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 -- start start 2024-05-14T17:02:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.911+0000 7f3751b89b80 1 Processor -- start 2024-05-14T17:02:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fd01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:38.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fd0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fc9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b18fc6bf0 con 0x555b18fb1fa0 2024-05-14T17:02:38.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b18fc6520 con 0x555b18fb28c0 2024-05-14T17:02:38.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b18fc72c0 con 0x555b18facdc0 2024-05-14T17:02:38.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 10 client.0 _ll_register_callbacks cb 0x555b18e95900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:38.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fd0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:38.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fd0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54840/0 (socket says 172.21.15.99:54840) 2024-05-14T17:02:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 -- 172.21.15.99:0/937688047 learned_addr learned my addr 172.21.15.99:0/937688047 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:38.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374508f700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fd01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:38.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3745890700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fc9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:38.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 -- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 msgr2=0x555b18fd01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:38.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fd01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:38.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 -- 172.21.15.99:0/937688047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 msgr2=0x555b18fc9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:38.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fc9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 -- 172.21.15.99:0/937688047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b18fc9ac0 con 0x555b18fb1fa0 2024-05-14T17:02:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f374488e700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fd0730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3730020040 tx=0x7f37300095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77aa76dad252dba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:39.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3728ff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:02:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3728ff9700 1 -- 172.21.15.99:0/937688047 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373002a060 con 0x555b18fb1fa0 2024-05-14T17:02:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3728ff9700 1 -- 172.21.15.99:0/937688047 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f3730015620 con 0x555b18fb1fa0 2024-05-14T17:02:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3730014c70 con 0x555b18fb1fa0 2024-05-14T17:02:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3728ff9700 1 -- 172.21.15.99:0/937688047 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3730022070 con 0x555b18fb1fa0 2024-05-14T17:02:39.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 10 client.15006 fetch_fsmap learned FSMap version 25 2024-05-14T17:02:39.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.912+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b19035da0 con 0x555b18fb1fa0 2024-05-14T17:02:39.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3728ff9700 1 -- 172.21.15.99:0/937688047 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3730014c70 con 0x555b18fb1fa0 2024-05-14T17:02:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3728ff9700 1 -- 172.21.15.99:0/937688047 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f37300308e0 con 0x555b18fb1fa0 2024-05-14T17:02:39.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3751b89b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:02:39.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3751b89b80 10 client.15006 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T17:02:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3751b89b80 20 client.15006 populate_metadata read hostname 'smithi099' 2024-05-14T17:02:39.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3751b89b80 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b1903a520 0x555b19039230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3751b89b80 4 client.15006 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:02:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "status"}) -- 0x555b18f06450 con 0x555b1903a520 2024-05-14T17:02:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.913+0000 7f374508f700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b1903a520 0x555b19039230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f374508f700 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b1903a520 0x555b19039230 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7953d35af837fe server_cookie=7e3675d8da0de56f in_seq=0 out_seq=0 2024-05-14T17:02:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3728ff9700 10 client.15006 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:02:39.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3728ff9700 1 -- 172.21.15.99:0/937688047 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555b18f06450 con 0x555b1903a520 2024-05-14T17:02:39.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3728ff9700 10 client.15006 handle_command_reply: tid=0 2024-05-14T17:02:39.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 client.15006 shutdown 2024-05-14T17:02:39.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 msgr2=0x555b18fd0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fd0730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3730020040 tx=0x7f37300095d0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 shutdown_connections 2024-05-14T17:02:39.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b18fb28c0 0x555b18fc9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b18facdc0 0x555b18fd01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x555b1903a520 0x555b19039230 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 --2- 172.21.15.99:0/937688047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b18fb1fa0 0x555b18fd0730 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3728ff9700 0 client.15006 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:02:39.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 >> 172.21.15.99:0/937688047 conn(0x555b18ed9d60 msgr2=0x555b18fb3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 shutdown_connections 2024-05-14T17:02:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 1 -- 172.21.15.99:0/937688047 wait complete. 2024-05-14T17:02:39.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.914+0000 7f3751b89b80 20 client.15006 trim_cache size 0 max 16384 2024-05-14T17:02:39.901 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:02:39.909 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:02:39.917 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-05-14T17:02:39.924 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24463, 2024-05-14T17:02:39.934 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:02:39.944 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:02:39.955 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:02:39.985 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 143.86041838, 2024-05-14T17:02:39.994 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-05-14T17:02:40.002 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 82, 2024-05-14T17:02:40.030 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T17:02:40.039 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 170.94295008500001 2024-05-14T17:02:40.050 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:02:40.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f372c0b2880 msgr2=0x7f372c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f372c0b2880 0x7f372c0b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f373c00b580 tx=0x7f373c009f10 comp rx=0 tx=0).stop 2024-05-14T17:02:40.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 msgr2=0x7f3740199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740199930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f373800cca0 tx=0x7f373800a060 comp rx=0 tx=0).stop 2024-05-14T17:02:40.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 shutdown_connections 2024-05-14T17:02:40.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f372c0b2880 0x7f372c0b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740109630 0x7f3740199930 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3740101ab0 0x7f374019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 --2- 172.21.15.99:0/2415567489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3740102470 0x7f374019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 >> 172.21.15.99:0/2415567489 conn(0x7f3740078a40 msgr2=0x7f37400fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:40.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 shutdown_connections 2024-05-14T17:02:40.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:35.915+0000 7f37472f3700 1 -- 172.21.15.99:0/2415567489 wait complete. 2024-05-14T17:02:40.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:36 smithi099 ceph-mon[31851]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:40.225 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:36 smithi193 ceph-mon[35962]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:40.235 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:38 smithi193 ceph-mon[35962]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:40.244 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:36 smithi150 ceph-mon[36204]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:40.253 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:38 smithi150 ceph-mon[36204]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:40.263 INFO:tasks.cephadm:Setup complete, yielding 2024-05-14T17:02:40.284 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T17:02:40.299 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:02:40.306 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-14T17:02:40.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:38 smithi099 ceph-mon[31851]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:40.347 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:40 smithi099 ceph-mon[31851]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:02:40.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:40 smithi150 ceph-mon[36204]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:02:40.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:40 smithi193 ceph-mon[35962]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:02:40.571 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:42.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 -- 172.21.15.99:0/352922125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd0104890 msgr2=0x7fecd0104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 --2- 172.21.15.99:0/352922125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd0104890 0x7fecd0104cf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009990 tx=0x7fecc4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 -- 172.21.15.99:0/352922125 shutdown_connections 2024-05-14T17:02:42.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 --2- 172.21.15.99:0/352922125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0105230 0x7fecd010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 --2- 172.21.15.99:0/352922125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecd01018b0 0x7fecd0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 --2- 172.21.15.99:0/352922125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd0104890 0x7fecd0104cf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 -- 172.21.15.99:0/352922125 >> 172.21.15.99:0/352922125 conn(0x7fecd00fd210 msgr2=0x7fecd00ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 -- 172.21.15.99:0/352922125 shutdown_connections 2024-05-14T17:02:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.139+0000 7fecd546f700 1 -- 172.21.15.99:0/352922125 wait complete. 2024-05-14T17:02:42.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 Processor -- start 2024-05-14T17:02:42.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 -- start start 2024-05-14T17:02:42.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 0x7fecd0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0104890 0x7fecd0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecd0105230 0x7fecd01a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fecd010f0d0 con 0x7fecd01018b0 2024-05-14T17:02:42.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fecd010ef50 con 0x7fecd0105230 2024-05-14T17:02:42.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecd546f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fecd010f250 con 0x7fecd0104890 2024-05-14T17:02:42.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 0x7fecd0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecceffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 0x7fecd0199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54862/0 (socket says 172.21.15.99:54862) 2024-05-14T17:02:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.140+0000 7fecceffd700 1 -- 172.21.15.99:0/3037245329 learned_addr learned my addr 172.21.15.99:0/3037245329 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:42.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7feccf7fe700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecd0105230 0x7fecd01a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecce7fc700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0104890 0x7fecd0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecceffd700 1 -- 172.21.15.99:0/3037245329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0104890 msgr2=0x7fecd0199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecceffd700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0104890 0x7fecd0199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecceffd700 1 -- 172.21.15.99:0/3037245329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecd0105230 msgr2=0x7fecd01a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecceffd700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecd0105230 0x7fecd01a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecceffd700 1 -- 172.21.15.99:0/3037245329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd0102700 con 0x7fecd01018b0 2024-05-14T17:02:42.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecce7fc700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0104890 0x7fecd0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:42.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecceffd700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 0x7fecd0199720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c9a0 tx=0x7fecc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:42.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004220 con 0x7fecd01018b0 2024-05-14T17:02:42.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd01029f0 con 0x7fecd01018b0 2024-05-14T17:02:42.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc0007d10 con 0x7fecd01018b0 2024-05-14T17:02:42.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0003d20 con 0x7fecd01018b0 2024-05-14T17:02:42.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.141+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd0103000 con 0x7fecd01018b0 2024-05-14T17:02:42.612 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:42 smithi150 ceph-mon[36204]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:42.619 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:42 smithi193 ceph-mon[35962]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:42.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.142+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd006a230 con 0x7fecd01018b0 2024-05-14T17:02:42.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.143+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fecc0007860 con 0x7fecd01018b0 2024-05-14T17:02:42.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.144+0000 7fecb7fff700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fecb80b2900 0x7fecb80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.144+0000 7fecce7fc700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fecb80b2900 0x7fecb80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.144+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fecc00e1d60 con 0x7fecd01018b0 2024-05-14T17:02:42.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.145+0000 7fecce7fc700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fecb80b2900 0x7fecb80b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fecc400b5c0 tx=0x7fecc4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.149+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecc0014030 con 0x7fecd01018b0 2024-05-14T17:02:42.784 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:42 smithi099 ceph-mon[31851]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:42.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.374+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fecd010a820 con 0x7fecb80b2900 2024-05-14T17:02:42.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.375+0000 7fecb7fff700 1 -- 172.21.15.99:0/3037245329 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fecd010a820 con 0x7fecb80b2900 2024-05-14T17:02:42.810 INFO:teuthology.orchestra.run.smithi099.stdout:Backend: cephadm 2024-05-14T17:02:42.818 INFO:teuthology.orchestra.run.smithi099.stdout:Available: Yes 2024-05-14T17:02:42.827 INFO:teuthology.orchestra.run.smithi099.stdout:Paused: No 2024-05-14T17:02:42.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fecb80b2900 msgr2=0x7fecb80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fecb80b2900 0x7fecb80b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fecc400b5c0 tx=0x7fecc4005fd0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 msgr2=0x7fecd0199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 0x7fecd0199720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c9a0 tx=0x7fecc000ccb0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 shutdown_connections 2024-05-14T17:02:42.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fecb80b2900 0x7fecb80b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecd0105230 0x7fecd01a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd01018b0 0x7fecd0199720 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 --2- 172.21.15.99:0/3037245329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd0104890 0x7fecd0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 >> 172.21.15.99:0/3037245329 conn(0x7fecd00fd210 msgr2=0x7fecd0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:42.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 shutdown_connections 2024-05-14T17:02:42.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:42.378+0000 7fecd546f700 1 -- 172.21.15.99:0/3037245329 wait complete. 2024-05-14T17:02:43.134 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-14T17:02:43.421 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:44.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:44 smithi099 ceph-mon[31851]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:44.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:44 smithi099 ceph-mon[31851]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:44.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:44 smithi150 ceph-mon[36204]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:44.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:44 smithi150 ceph-mon[36204]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:44.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:44 smithi193 ceph-mon[35962]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:44.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:44 smithi193 ceph-mon[35962]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:45.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.006+0000 7f665ca6b700 1 -- 172.21.15.99:0/1716573213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658102e30 msgr2=0x7f6658103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:45.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.006+0000 7f665ca6b700 1 --2- 172.21.15.99:0/1716573213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658102e30 0x7f6658103290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6648009990 tx=0x7f6648009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.006+0000 7f665ca6b700 1 -- 172.21.15.99:0/1716573213 shutdown_connections 2024-05-14T17:02:45.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.006+0000 7f665ca6b700 1 --2- 172.21.15.99:0/1716573213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66581037d0 0x7f665810a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.006+0000 7f665ca6b700 1 --2- 172.21.15.99:0/1716573213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6658101c30 0x7f6658102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.006+0000 7f665ca6b700 1 --2- 172.21.15.99:0/1716573213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658102e30 0x7f6658103290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.007+0000 7f665ca6b700 1 -- 172.21.15.99:0/1716573213 >> 172.21.15.99:0/1716573213 conn(0x7f66580fd1b0 msgr2=0x7f66580ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:45.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.007+0000 7f665ca6b700 1 -- 172.21.15.99:0/1716573213 shutdown_connections 2024-05-14T17:02:45.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.007+0000 7f665ca6b700 1 -- 172.21.15.99:0/1716573213 wait complete. 2024-05-14T17:02:45.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.007+0000 7f665ca6b700 1 Processor -- start 2024-05-14T17:02:45.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.007+0000 7f665ca6b700 1 -- start start 2024-05-14T17:02:45.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 0x7f66581996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:45.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658102e30 0x7f6658199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:45.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66581037d0 0x7f66581a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:45.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f665810f0f0 con 0x7f6658101c30 2024-05-14T17:02:45.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f665810ef70 con 0x7f66581037d0 2024-05-14T17:02:45.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f665810f270 con 0x7f6658102e30 2024-05-14T17:02:45.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 0x7f66581996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 0x7f66581996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38892/0 (socket says 172.21.15.99:38892) 2024-05-14T17:02:45.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 -- 172.21.15.99:0/382399738 learned_addr learned my addr 172.21.15.99:0/382399738 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:45.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f6655d9b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658102e30 0x7f6658199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:45.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f6656d9d700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66581037d0 0x7f66581a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:45.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 -- 172.21.15.99:0/382399738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658102e30 msgr2=0x7f6658199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:45.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658102e30 0x7f6658199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 -- 172.21.15.99:0/382399738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66581037d0 msgr2=0x7f66581a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:45.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66581037d0 0x7f66581a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 -- 172.21.15.99:0/382399738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66581a1350 con 0x7f6658101c30 2024-05-14T17:02:45.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f6656d9d700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66581037d0 0x7f66581a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:45.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665659c700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 0x7f66581996f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f664000b530 tx=0x7f664000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:45.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640015020 con 0x7f6658101c30 2024-05-14T17:02:45.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6640011ce0 con 0x7f6658101c30 2024-05-14T17:02:45.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66581a1640 con 0x7f6658101c30 2024-05-14T17:02:45.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.008+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640010d60 con 0x7f6658101c30 2024-05-14T17:02:45.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.009+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66581a1c00 con 0x7f6658101c30 2024-05-14T17:02:45.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.010+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665806a230 con 0x7f6658101c30 2024-05-14T17:02:45.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.011+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6640018020 con 0x7f6658101c30 2024-05-14T17:02:45.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.011+0000 7f664f7fe700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f66440b2880 0x7f66440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:45.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.012+0000 7f6655d9b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f66440b2880 0x7f66440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:45.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.012+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f66400e1fd0 con 0x7f6658101c30 2024-05-14T17:02:45.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.012+0000 7f6655d9b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f66440b2880 0x7f66440b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f664800b5c0 tx=0x7f6648005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:45.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.015+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f664001f020 con 0x7f6658101c30 2024-05-14T17:02:45.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.226+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6658063980 con 0x7f66440b2880 2024-05-14T17:02:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.240+0000 7f664f7fe700 1 -- 172.21.15.99:0/382399738 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6658063980 con 0x7f66440b2880 2024-05-14T17:02:45.620 INFO:teuthology.orchestra.run.smithi099.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-14T17:02:45.628 INFO:teuthology.orchestra.run.smithi099.stdout:mds.a smithi099 running (2m) 37s ago 2m 14.4M - 19.0.0-3687-g9b638668 c922e207469e bf58256064fa 2024-05-14T17:02:45.636 INFO:teuthology.orchestra.run.smithi099.stdout:mds.b smithi150 running (2m) 38s ago 2m 21.3M - 19.0.0-3687-g9b638668 c922e207469e d5eb5f5c43f6 2024-05-14T17:02:45.645 INFO:teuthology.orchestra.run.smithi099.stdout:mds.c smithi193 running (3m) 38s ago 3m 21.6M - 19.0.0-3687-g9b638668 c922e207469e d455cbdc36f6 2024-05-14T17:02:45.653 INFO:teuthology.orchestra.run.smithi099.stdout:mds.d smithi099 running (3m) 37s ago 3m 14.8M - 19.0.0-3687-g9b638668 c922e207469e d9410889b826 2024-05-14T17:02:45.662 INFO:teuthology.orchestra.run.smithi099.stdout:mds.e smithi150 running (3m) 38s ago 3m 17.0M - 19.0.0-3687-g9b638668 c922e207469e c1b79095d2af 2024-05-14T17:02:45.671 INFO:teuthology.orchestra.run.smithi099.stdout:mds.f smithi193 running (3m) 38s ago 3m 19.7M - 19.0.0-3687-g9b638668 c922e207469e ed371fb0d113 2024-05-14T17:02:45.681 INFO:teuthology.orchestra.run.smithi099.stdout:mds.g smithi099 running (2m) 37s ago 2m 14.7M - 19.0.0-3687-g9b638668 c922e207469e c7a64b727342 2024-05-14T17:02:45.688 INFO:teuthology.orchestra.run.smithi099.stdout:mds.h smithi150 running (3m) 38s ago 3m 20.8M - 19.0.0-3687-g9b638668 c922e207469e 2dd5281220ac 2024-05-14T17:02:45.713 INFO:teuthology.orchestra.run.smithi099.stdout:mds.i smithi193 running (3m) 38s ago 3m 28.2M - 19.0.0-3687-g9b638668 c922e207469e 3d536fb5be46 2024-05-14T17:02:45.735 INFO:teuthology.orchestra.run.smithi099.stdout:mds.j smithi099 running (3m) 37s ago 3m 16.3M - 19.0.0-3687-g9b638668 c922e207469e 19663200bb62 2024-05-14T17:02:45.742 INFO:teuthology.orchestra.run.smithi099.stdout:mds.k smithi150 running (3m) 38s ago 3m 14.9M - 19.0.0-3687-g9b638668 c922e207469e 2b9040597b5e 2024-05-14T17:02:45.751 INFO:teuthology.orchestra.run.smithi099.stdout:mds.l smithi193 running (3m) 38s ago 3m 18.5M - 19.0.0-3687-g9b638668 c922e207469e 291487fb2beb 2024-05-14T17:02:45.759 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.x smithi099 *:9283,8765 running (10m) 37s ago 10m 529M - 19.0.0-3687-g9b638668 c922e207469e f9791659271c 2024-05-14T17:02:45.767 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.y smithi150 *:8443,8765 running (8m) 38s ago 8m 467M - 19.0.0-3687-g9b638668 c922e207469e 6b35e6d20c56 2024-05-14T17:02:45.776 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.z smithi193 *:8443,8765 running (8m) 38s ago 8m 466M - 19.0.0-3687-g9b638668 c922e207469e b02a4c2b1909 2024-05-14T17:02:45.784 INFO:teuthology.orchestra.run.smithi099.stdout:mon.a smithi099 running (10m) 37s ago 10m 51.7M 2048M 19.0.0-3687-g9b638668 c922e207469e 6f2cc53d7976 2024-05-14T17:02:45.792 INFO:teuthology.orchestra.run.smithi099.stdout:mon.b smithi150 running (8m) 38s ago 8m 42.1M 2048M 19.0.0-3687-g9b638668 c922e207469e a6945f6732f7 2024-05-14T17:02:45.802 INFO:teuthology.orchestra.run.smithi099.stdout:mon.c smithi193 running (8m) 38s ago 8m 42.4M 2048M 19.0.0-3687-g9b638668 c922e207469e 2e8dd14d2223 2024-05-14T17:02:45.811 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 smithi099 running (8m) 37s ago 8m 58.2M 4272M 19.0.0-3687-g9b638668 c922e207469e 3034211c3be0 2024-05-14T17:02:45.820 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 smithi150 running (7m) 38s ago 7m 48.9M 4272M 19.0.0-3687-g9b638668 c922e207469e 508e949825b8 2024-05-14T17:02:45.829 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 smithi193 running (7m) 38s ago 7m 50.0M 4272M 19.0.0-3687-g9b638668 c922e207469e 3bbcaab0f889 2024-05-14T17:02:45.838 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 smithi099 running (7m) 37s ago 7m 48.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 470c8c9a8bc4 2024-05-14T17:02:45.848 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 smithi150 running (6m) 38s ago 6m 45.6M 4272M 19.0.0-3687-g9b638668 c922e207469e 6034dcd46af2 2024-05-14T17:02:45.858 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 smithi193 running (6m) 38s ago 6m 56.0M 4272M 19.0.0-3687-g9b638668 c922e207469e 351fd60d2be8 2024-05-14T17:02:45.867 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 smithi099 running (5m) 37s ago 5m 56.7M 4272M 19.0.0-3687-g9b638668 c922e207469e b2c2d9927caa 2024-05-14T17:02:45.899 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 smithi150 running (5m) 38s ago 5m 47.7M 4272M 19.0.0-3687-g9b638668 c922e207469e b6fc71cacc1c 2024-05-14T17:02:45.934 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 smithi193 running (5m) 38s ago 5m 55.4M 4272M 19.0.0-3687-g9b638668 c922e207469e 88e88f839ec0 2024-05-14T17:02:45.943 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 smithi099 running (4m) 37s ago 4m 45.8M 4272M 19.0.0-3687-g9b638668 c922e207469e 6570294a4305 2024-05-14T17:02:45.951 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 smithi150 running (4m) 38s ago 4m 47.7M 4272M 19.0.0-3687-g9b638668 c922e207469e 5123cb3fc54c 2024-05-14T17:02:45.958 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 smithi193 running (3m) 38s ago 3m 46.3M 4272M 19.0.0-3687-g9b638668 c922e207469e fc020acaf9c9 2024-05-14T17:02:45.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f66440b2880 msgr2=0x7f66440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:45.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f66440b2880 0x7f66440b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f664800b5c0 tx=0x7f6648005fd0 comp rx=0 tx=0).stop 2024-05-14T17:02:45.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 msgr2=0x7f66581996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:45.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 0x7f66581996f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f664000b530 tx=0x7f664000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 shutdown_connections 2024-05-14T17:02:46.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f66440b2880 0x7f66440b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66581037d0 0x7f66581a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6658101c30 0x7f66581996f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 --2- 172.21.15.99:0/382399738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6658102e30 0x7f6658199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:46.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 >> 172.21.15.99:0/382399738 conn(0x7f66580fd1b0 msgr2=0x7f66581060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:46.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 shutdown_connections 2024-05-14T17:02:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:45.242+0000 7f665ca6b700 1 -- 172.21.15.99:0/382399738 wait complete. 2024-05-14T17:02:46.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-14T17:02:46.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:46 smithi099 ceph-mon[31851]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:46.271 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:46 smithi099 ceph-mon[31851]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:46.427 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:46.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:46 smithi150 ceph-mon[36204]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:46.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:46 smithi150 ceph-mon[36204]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:46.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:46 smithi193 ceph-mon[35962]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:46.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:46 smithi193 ceph-mon[35962]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:48.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 -- 172.21.15.99:0/849473255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890101610 msgr2=0x7f0890101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:48.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 --2- 172.21.15.99:0/849473255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890101610 0x7f0890101a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0880009990 tx=0x7f0880009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 -- 172.21.15.99:0/849473255 shutdown_connections 2024-05-14T17:02:48.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 --2- 172.21.15.99:0/849473255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 0x7f089010bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 --2- 172.21.15.99:0/849473255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0890104b90 0x7f0890104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 --2- 172.21.15.99:0/849473255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890101610 0x7f0890101a70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 -- 172.21.15.99:0/849473255 >> 172.21.15.99:0/849473255 conn(0x7f08900fd1b0 msgr2=0x7f08900ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:48.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 -- 172.21.15.99:0/849473255 shutdown_connections 2024-05-14T17:02:48.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.014+0000 7f0894ae2700 1 -- 172.21.15.99:0/849473255 wait complete. 2024-05-14T17:02:48.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.015+0000 7f0894ae2700 1 Processor -- start 2024-05-14T17:02:48.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.015+0000 7f0894ae2700 1 -- start start 2024-05-14T17:02:48.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.015+0000 7f0894ae2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0890101610 0x7f0890199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 0x7f0890199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 0x7f08901a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:48.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f089010f0c0 con 0x7f0890104b90 2024-05-14T17:02:48.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f089010ef40 con 0x7f0890101fb0 2024-05-14T17:02:48.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f089010f240 con 0x7f0890101610 2024-05-14T17:02:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 0x7f08901a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 0x7f08901a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38910/0 (socket says 172.21.15.99:38910) 2024-05-14T17:02:48.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 -- 172.21.15.99:0/2113736811 learned_addr learned my addr 172.21.15.99:0/2113736811 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:48.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088e59c700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0890101610 0x7f0890199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088dd9b700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 0x7f0890199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 -- 172.21.15.99:0/2113736811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0890101610 msgr2=0x7f0890199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:48.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0890101610 0x7f0890199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 -- 172.21.15.99:0/2113736811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 msgr2=0x7f0890199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:48.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 0x7f0890199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 -- 172.21.15.99:0/2113736811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08901a1390 con 0x7f0890104b90 2024-05-14T17:02:48.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088dd9b700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 0x7f0890199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:48.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f088ed9d700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 0x7f08901a0cf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f088800b9a0 tx=0x7f088800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:48.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888019070 con 0x7f0890104b90 2024-05-14T17:02:48.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088800cde0 con 0x7f0890104b90 2024-05-14T17:02:48.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08901a1680 con 0x7f0890104b90 2024-05-14T17:02:48.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888007370 con 0x7f0890104b90 2024-05-14T17:02:48.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.016+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08901a1c10 con 0x7f0890104b90 2024-05-14T17:02:48.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:48 smithi150 ceph-mon[36204]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.017+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089006a230 con 0x7f0890104b90 2024-05-14T17:02:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.019+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f088801c070 con 0x7f0890104b90 2024-05-14T17:02:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.019+0000 7f08877fe700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f087c0b2900 0x7f087c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:48.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.019+0000 7f088e59c700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f087c0b2900 0x7f087c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:48.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.019+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f08880e1e20 con 0x7f0890104b90 2024-05-14T17:02:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.020+0000 7f088e59c700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f087c0b2900 0x7f087c0b4dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f089006ba10 tx=0x7f087800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.023+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0888023070 con 0x7f0890104b90 2024-05-14T17:02:48.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.235+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0890063980 con 0x7f087c0b2900 2024-05-14T17:02:48.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.242+0000 7f08877fe700 1 -- 172.21.15.99:0/2113736811 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0890063980 con 0x7f087c0b2900 2024-05-14T17:02:48.636 INFO:teuthology.orchestra.run.smithi099.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-14T17:02:48.642 INFO:teuthology.orchestra.run.smithi099.stdout:mds.all 12/12 41s ago 3m smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T17:02:48.651 INFO:teuthology.orchestra.run.smithi099.stdout:mgr 3/3 41s ago 8m smithi099=x;smithi150=y;smithi193=z;count:3 2024-05-14T17:02:48.658 INFO:teuthology.orchestra.run.smithi099.stdout:mon 3/3 41s ago 9m smithi099:172.21.15.99=a;smithi150:172.21.15.150=b;smithi193:172.21.15.193=c;count:3 2024-05-14T17:02:48.664 INFO:teuthology.orchestra.run.smithi099.stdout:osd 12 41s ago - 2024-05-14T17:02:48.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f087c0b2900 msgr2=0x7f087c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:48.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f087c0b2900 0x7f087c0b4dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f089006ba10 tx=0x7f087800b410 comp rx=0 tx=0).stop 2024-05-14T17:02:48.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 msgr2=0x7f08901a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:48.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 0x7f08901a0cf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f088800b9a0 tx=0x7f088800bcb0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 shutdown_connections 2024-05-14T17:02:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f087c0b2900 0x7f087c0b4dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0890104b90 0x7f08901a0cf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0890101610 0x7f0890199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 --2- 172.21.15.99:0/2113736811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101fb0 0x7f0890199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 >> 172.21.15.99:0/2113736811 conn(0x7f08900fd1b0 msgr2=0x7f0890107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 shutdown_connections 2024-05-14T17:02:48.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:48.244+0000 7f0894ae2700 1 -- 172.21.15.99:0/2113736811 wait complete. 2024-05-14T17:02:48.801 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:48 smithi099 ceph-mon[31851]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:48.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:48 smithi193 ceph-mon[35962]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:48.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-14T17:02:49.255 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:49.277 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:49 smithi099 ceph-mon[31851]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:49.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:49 smithi150 ceph-mon[36204]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:49 smithi193 ceph-mon[35962]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:50.425 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:50 smithi099 ceph-mon[31851]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:02:50.467 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:50 smithi150 ceph-mon[36204]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:02:50.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:50 smithi193 ceph-mon[35962]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:02:50.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 -- 172.21.15.99:0/1908925504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 msgr2=0x7fad6810b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:50.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 --2- 172.21.15.99:0/1908925504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad6810b6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad6000b0a0 tx=0x7fad6000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:02:50.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 -- 172.21.15.99:0/1908925504 shutdown_connections 2024-05-14T17:02:51.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 --2- 172.21.15.99:0/1908925504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad6810b6f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 --2- 172.21.15.99:0/1908925504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68104020 0x7fad68106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 --2- 172.21.15.99:0/1908925504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68106950 0x7fad68108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 -- 172.21.15.99:0/1908925504 >> 172.21.15.99:0/1908925504 conn(0x7fad680fda30 msgr2=0x7fad680ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:51.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 -- 172.21.15.99:0/1908925504 shutdown_connections 2024-05-14T17:02:51.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.965+0000 7fad6d149700 1 -- 172.21.15.99:0/1908925504 wait complete. 2024-05-14T17:02:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 Processor -- start 2024-05-14T17:02:51.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 -- start start 2024-05-14T17:02:51.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68104020 0x7fad68199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68106950 0x7fad68199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:51.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad681a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:51.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810f070 con 0x7fad68109280 2024-05-14T17:02:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810eef0 con 0x7fad68106950 2024-05-14T17:02:51.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6d149700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810f1f0 con 0x7fad68104020 2024-05-14T17:02:51.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad681a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:51.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6759e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad681a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38938/0 (socket says 172.21.15.99:38938) 2024-05-14T17:02:51.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad6759e700 1 -- 172.21.15.99:0/2495711787 learned_addr learned my addr 172.21.15.99:0/2495711787 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:51.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad66d9d700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68104020 0x7fad68199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:51.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.966+0000 7fad5ffff700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68106950 0x7fad68199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:51.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6759e700 1 -- 172.21.15.99:0/2495711787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68104020 msgr2=0x7fad68199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:51.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6759e700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68104020 0x7fad68199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6759e700 1 -- 172.21.15.99:0/2495711787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68106950 msgr2=0x7fad68199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:51.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6759e700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68106950 0x7fad68199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6759e700 1 -- 172.21.15.99:0/2495711787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad68101e10 con 0x7fad68109280 2024-05-14T17:02:51.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad66d9d700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68104020 0x7fad68199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:51.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad5ffff700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68106950 0x7fad68199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:51.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6759e700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad681a0c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad6000ced0 tx=0x7fad6000cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:51.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60021070 con 0x7fad68109280 2024-05-14T17:02:51.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad681020a0 con 0x7fad68109280 2024-05-14T17:02:51.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad60004020 con 0x7fad68109280 2024-05-14T17:02:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6001d690 con 0x7fad68109280 2024-05-14T17:02:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.967+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681026b0 con 0x7fad68109280 2024-05-14T17:02:51.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.968+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6806a230 con 0x7fad68109280 2024-05-14T17:02:51.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.970+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fad600041d0 con 0x7fad68109280 2024-05-14T17:02:51.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.970+0000 7fad64d99700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fad540b2880 0x7fad540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:51.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.970+0000 7fad66d9d700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fad540b2880 0x7fad540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:51.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.970+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fad6000f020 con 0x7fad68109280 2024-05-14T17:02:51.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.971+0000 7fad66d9d700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fad540b2880 0x7fad540b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad580099d0 tx=0x7fad58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:51.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:50.974+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad6002a020 con 0x7fad68109280 2024-05-14T17:02:51.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.182+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fad680ff100 con 0x7fad540b2880 2024-05-14T17:02:51.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.183+0000 7fad64d99700 1 -- 172.21.15.99:0/2495711787 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fad680ff100 con 0x7fad540b2880 2024-05-14T17:02:51.558 INFO:teuthology.orchestra.run.smithi099.stdout:HOST ADDR LABELS STATUS 2024-05-14T17:02:51.565 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 172.21.15.99 2024-05-14T17:02:51.571 INFO:teuthology.orchestra.run.smithi099.stdout:smithi150 172.21.15.150 2024-05-14T17:02:51.579 INFO:teuthology.orchestra.run.smithi099.stdout:smithi193 172.21.15.193 2024-05-14T17:02:51.586 INFO:teuthology.orchestra.run.smithi099.stdout:3 hosts in cluster 2024-05-14T17:02:51.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.185+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fad540b2880 msgr2=0x7fad540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:51.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.185+0000 7fad6d149700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fad540b2880 0x7fad540b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad580099d0 tx=0x7fad58008040 comp rx=0 tx=0).stop 2024-05-14T17:02:51.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.185+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 msgr2=0x7fad681a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.185+0000 7fad6d149700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad681a0c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad6000ced0 tx=0x7fad6000cfb0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 shutdown_connections 2024-05-14T17:02:51.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fad540b2880 0x7fad540b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad68109280 0x7fad681a0c20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad68104020 0x7fad68199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 --2- 172.21.15.99:0/2495711787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad68106950 0x7fad68199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:51.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 >> 172.21.15.99:0/2495711787 conn(0x7fad680fda30 msgr2=0x7fad681048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 shutdown_connections 2024-05-14T17:02:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:51.186+0000 7fad6d149700 1 -- 172.21.15.99:0/2495711787 wait complete. 2024-05-14T17:02:51.777 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-14T17:02:52.025 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:52.378 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:52 smithi099 ceph-mon[31851]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:02:52.402 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:52 smithi099 ceph-mon[31851]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:52 smithi150 ceph-mon[36204]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:02:52.493 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:52 smithi150 ceph-mon[36204]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:52 smithi193 ceph-mon[35962]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:02:52.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:52 smithi193 ceph-mon[35962]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:53.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 -- 172.21.15.99:0/1671904860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c1046c0 msgr2=0x7f714c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 --2- 172.21.15.99:0/1671904860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c1046c0 0x7f714c104b20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f713c0099c0 tx=0x7f713c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 -- 172.21.15.99:0/1671904860 shutdown_connections 2024-05-14T17:02:53.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 --2- 172.21.15.99:0/1671904860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c105060 0x7f714c10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 --2- 172.21.15.99:0/1671904860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f714c1034c0 0x7f714c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 --2- 172.21.15.99:0/1671904860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c1046c0 0x7f714c104b20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.707+0000 7f71521a0700 1 -- 172.21.15.99:0/1671904860 >> 172.21.15.99:0/1671904860 conn(0x7f714c0fea80 msgr2=0x7f714c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:53.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 -- 172.21.15.99:0/1671904860 shutdown_connections 2024-05-14T17:02:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 -- 172.21.15.99:0/1671904860 wait complete. 2024-05-14T17:02:53.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 Processor -- start 2024-05-14T17:02:53.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 -- start start 2024-05-14T17:02:53.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f714c1034c0 0x7f714c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c1046c0 0x7f714c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 0x7f714c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10f000 con 0x7f714c105060 2024-05-14T17:02:53.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10ee80 con 0x7f714c1034c0 2024-05-14T17:02:53.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.708+0000 7f71521a0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10f180 con 0x7f714c1046c0 2024-05-14T17:02:53.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 0x7f714c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715119e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f714c1034c0 0x7f714c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 0x7f714c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38966/0 (socket says 172.21.15.99:38966) 2024-05-14T17:02:54.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715099d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c1046c0 0x7f714c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:54.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 -- 172.21.15.99:0/1902906722 learned_addr learned my addr 172.21.15.99:0/1902906722 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:54.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 -- 172.21.15.99:0/1902906722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c1046c0 msgr2=0x7f714c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c1046c0 0x7f714c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:54.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 -- 172.21.15.99:0/1902906722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f714c1034c0 msgr2=0x7f714c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f714c1034c0 0x7f714c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:54.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 -- 172.21.15.99:0/1902906722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c1a1240 con 0x7f714c105060 2024-05-14T17:02:54.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f715199f700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 0x7f714c1a0ba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7144007f10 tx=0x7f714400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144004bb0 con 0x7f714c105060 2024-05-14T17:02:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c0757d0 con 0x7f714c105060 2024-05-14T17:02:54.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7144004d50 con 0x7f714c105060 2024-05-14T17:02:54.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440058e0 con 0x7f714c105060 2024-05-14T17:02:54.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.709+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c075d60 con 0x7f714c105060 2024-05-14T17:02:54.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:54 smithi193 ceph-mon[35962]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:54.833 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:54 smithi150 ceph-mon[36204]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:54.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.710+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c06a230 con 0x7f714c105060 2024-05-14T17:02:54.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.712+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f714401a020 con 0x7f714c105060 2024-05-14T17:02:54.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.712+0000 7f71427fc700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71380b2880 0x7f71380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:54.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.713+0000 7f715119e700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71380b2880 0x7f71380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.713+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f71440e1f30 con 0x7f714c105060 2024-05-14T17:02:55.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.713+0000 7f715119e700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71380b2880 0x7f71380b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f714c104520 tx=0x7f7148009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.716+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7144010070 con 0x7f714c105060 2024-05-14T17:02:55.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.929+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 --> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f714c063980 con 0x7f71380b2880 2024-05-14T17:02:55.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.931+0000 7f71427fc700 1 -- 172.21.15.99:0/1902906722 <== mgr.14152 v2:172.21.15.99:6800/2498052825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f714c063980 con 0x7f71380b2880 2024-05-14T17:02:55.158 INFO:teuthology.orchestra.run.smithi099.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-14T17:02:55.169 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T17:02:55.179 INFO:teuthology.orchestra.run.smithi099.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T17:02:55.196 INFO:teuthology.orchestra.run.smithi099.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-14T17:02:55.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71380b2880 msgr2=0x7f71380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:55.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71380b2880 0x7f71380b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f714c104520 tx=0x7f7148009190 comp rx=0 tx=0).stop 2024-05-14T17:02:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 msgr2=0x7f714c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:55.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 0x7f714c1a0ba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7144007f10 tx=0x7f714400d3b0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 shutdown_connections 2024-05-14T17:02:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f71380b2880 0x7f71380b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f714c105060 0x7f714c1a0ba0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f714c1034c0 0x7f714c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 --2- 172.21.15.99:0/1902906722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c1046c0 0x7f714c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 >> 172.21.15.99:0/1902906722 conn(0x7f714c0fea80 msgr2=0x7f714c1079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:55.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 shutdown_connections 2024-05-14T17:02:55.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:53.933+0000 7f71521a0700 1 -- 172.21.15.99:0/1902906722 wait complete. 2024-05-14T17:02:55.502 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:54 smithi099 ceph-mon[31851]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:55.513 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T17:02:55.529 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:02:55.538 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-14T17:02:55.547 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:55 smithi150 ceph-mon[36204]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:55.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:55 smithi193 ceph-mon[35962]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:55.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:55 smithi099 ceph-mon[31851]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:02:55.827 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:56.260 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:56 smithi099 ceph-mon[31851]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:56.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:56 smithi150 ceph-mon[36204]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:56.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:56 smithi193 ceph-mon[35962]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:02:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 -- 172.21.15.99:0/163461709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1024a0 msgr2=0x7f2c0c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:57.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 --2- 172.21.15.99:0/163461709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1024a0 0x7f2c0c1028c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c000099c0 tx=0x7f2c00009cd0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 -- 172.21.15.99:0/163461709 shutdown_connections 2024-05-14T17:02:57.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 --2- 172.21.15.99:0/163461709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c0c104040 0x7f2c0c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 --2- 172.21.15.99:0/163461709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1024a0 0x7f2c0c1028c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 --2- 172.21.15.99:0/163461709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 -- 172.21.15.99:0/163461709 >> 172.21.15.99:0/163461709 conn(0x7f2c0c0fda40 msgr2=0x7f2c0c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:57.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 -- 172.21.15.99:0/163461709 shutdown_connections 2024-05-14T17:02:57.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.362+0000 7f2c12d62700 1 -- 172.21.15.99:0/163461709 wait complete. 2024-05-14T17:02:57.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 Processor -- start 2024-05-14T17:02:57.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 -- start start 2024-05-14T17:02:57.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c0c1024a0 0x7f2c0c1063e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:57.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:57.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c104040 0x7f2c0c104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:57.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10e010 con 0x7f2c0c1036a0 2024-05-14T17:02:57.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10de90 con 0x7f2c0c1024a0 2024-05-14T17:02:57.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c12d62700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10e190 con 0x7f2c0c104040 2024-05-14T17:02:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:57.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41284/0 (socket says 172.21.15.99:41284) 2024-05-14T17:02:57.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 -- 172.21.15.99:0/1684239481 learned_addr learned my addr 172.21.15.99:0/1684239481 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:02:57.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c10afe700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c0c1024a0 0x7f2c0c1063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:57.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 -- 172.21.15.99:0/1684239481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c104040 msgr2=0x7f2c0c104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:57.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c104040 0x7f2c0c104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 -- 172.21.15.99:0/1684239481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c0c1024a0 msgr2=0x7f2c0c1063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:57.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c0c1024a0 0x7f2c0c1063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:57.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.363+0000 7f2c0bfff700 1 -- 172.21.15.99:0/1684239481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c105890 con 0x7f2c0c1036a0 2024-05-14T17:02:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.364+0000 7f2c0bfff700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c104a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b530 tx=0x7f2bf800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:57.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.364+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8015020 con 0x7f2c0c1036a0 2024-05-14T17:02:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.364+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf8011ce0 con 0x7f2c0c1036a0 2024-05-14T17:02:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.364+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c1a3990 con 0x7f2c0c1036a0 2024-05-14T17:02:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.364+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8010d20 con 0x7f2c0c1036a0 2024-05-14T17:02:57.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.364+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c1a3ed0 con 0x7f2c0c1036a0 2024-05-14T17:02:57.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.365+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c06a230 con 0x7f2c0c1036a0 2024-05-14T17:02:57.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.366+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2bf8018020 con 0x7f2c0c1036a0 2024-05-14T17:02:57.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.367+0000 7f2c09ffb700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2bfc0b2880 0x7f2bfc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:57.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.367+0000 7f2c10afe700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2bfc0b2880 0x7f2bfc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:57.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.367+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f2bf80e1ad0 con 0x7f2c0c1036a0 2024-05-14T17:02:58.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.368+0000 7f2c10afe700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2bfc0b2880 0x7f2bfc0b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000c00 tx=0x7f2c00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:58.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.372+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2bf801f020 con 0x7f2c0c1036a0 2024-05-14T17:02:58.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.620+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2c0c1a4810 con 0x7f2c0c1036a0 2024-05-14T17:02:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.620+0000 7f2c09ffb700 1 -- 172.21.15.99:0/1684239481 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3051 (secure 0 0 0) 0x7f2bf80a9800 con 0x7f2c0c1036a0 2024-05-14T17:02:58.052 INFO:teuthology.orchestra.run.smithi099.stdout:e25 2024-05-14T17:02:58.060 INFO:teuthology.orchestra.run.smithi099.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T17:02:58.067 INFO:teuthology.orchestra.run.smithi099.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T17:02:58.074 INFO:teuthology.orchestra.run.smithi099.stdout:legacy client fscid: 1 2024-05-14T17:02:58.081 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:58.089 INFO:teuthology.orchestra.run.smithi099.stdout:Filesystem 'cephfs' (1) 2024-05-14T17:02:58.096 INFO:teuthology.orchestra.run.smithi099.stdout:fs_name cephfs 2024-05-14T17:02:58.105 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 25 2024-05-14T17:02:58.111 INFO:teuthology.orchestra.run.smithi099.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-14T17:02:58.118 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-05-14T17:00:11.851265+0000 2024-05-14T17:02:58.124 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2024-05-14T17:00:30.962944+0000 2024-05-14T17:02:58.143 INFO:teuthology.orchestra.run.smithi099.stdout:tableserver 0 2024-05-14T17:02:58.177 INFO:teuthology.orchestra.run.smithi099.stdout:root 0 2024-05-14T17:02:58.212 INFO:teuthology.orchestra.run.smithi099.stdout:session_timeout 300 2024-05-14T17:02:58.220 INFO:teuthology.orchestra.run.smithi099.stdout:session_autoclose 300 2024-05-14T17:02:58.229 INFO:teuthology.orchestra.run.smithi099.stdout:max_file_size 1099511627776 2024-05-14T17:02:58.237 INFO:teuthology.orchestra.run.smithi099.stdout:max_xattr_size 65536 2024-05-14T17:02:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:required_client_features {} 2024-05-14T17:02:58.254 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure 0 2024-05-14T17:02:58.263 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure_osd_epoch 0 2024-05-14T17:02:58.272 INFO:teuthology.orchestra.run.smithi099.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T17:02:58.282 INFO:teuthology.orchestra.run.smithi099.stdout:max_mds 3 2024-05-14T17:02:58.291 INFO:teuthology.orchestra.run.smithi099.stdout:in 0,1,2 2024-05-14T17:02:58.300 INFO:teuthology.orchestra.run.smithi099.stdout:up {0=24463,1=24448,2=24443} 2024-05-14T17:02:58.309 INFO:teuthology.orchestra.run.smithi099.stdout:failed 2024-05-14T17:02:58.321 INFO:teuthology.orchestra.run.smithi099.stdout:damaged 2024-05-14T17:02:58.330 INFO:teuthology.orchestra.run.smithi099.stdout:stopped 2024-05-14T17:02:58.340 INFO:teuthology.orchestra.run.smithi099.stdout:data_pools [3] 2024-05-14T17:02:58.377 INFO:teuthology.orchestra.run.smithi099.stdout:metadata_pool 2 2024-05-14T17:02:58.409 INFO:teuthology.orchestra.run.smithi099.stdout:inline_data disabled 2024-05-14T17:02:58.428 INFO:teuthology.orchestra.run.smithi099.stdout:balancer 2024-05-14T17:02:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:bal_rank_mask -1 2024-05-14T17:02:58.446 INFO:teuthology.orchestra.run.smithi099.stdout:standby_count_wanted 0 2024-05-14T17:02:58.455 INFO:teuthology.orchestra.run.smithi099.stdout:qdb_cluster leader: 24463 members: 24448,24463,24443 2024-05-14T17:02:58.465 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.i{0:24463} state up:active seq 8 addr [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.474 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.f{0:24439} state up:standby-replay seq 1 addr [v2:172.21.15.193:6834/3113222034,v1:172.21.15.193:6835/3113222034] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.482 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c{1:24448} state up:active seq 13 addr [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.490 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.l{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/3825469703,v1:172.21.15.193:6833/3825469703] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.498 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b{2:24443} state up:active seq 9 addr [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.506 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.h{2:24413} state up:standby-replay seq 1 addr [v2:172.21.15.150:6836/1891176204,v1:172.21.15.150:6837/1891176204] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:58.526 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:58.535 INFO:teuthology.orchestra.run.smithi099.stdout:Standby daemons: 2024-05-14T17:02:58.543 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:02:58.551 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.99:6834/3118783767,v1:172.21.15.99:6835/3118783767] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.569 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.99:6836/2633931884,v1:172.21.15.99:6837/2633931884] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.628 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.g{-1:14601} state up:standby seq 1 addr [v2:172.21.15.99:6838/913097680,v1:172.21.15.99:6839/913097680] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.638 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a{-1:14607} state up:standby seq 1 addr [v2:172.21.15.99:6840/862670841,v1:172.21.15.99:6841/862670841] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.646 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.k{-1:24377} state up:standby seq 1 addr [v2:172.21.15.150:6832/308217426,v1:172.21.15.150:6833/308217426] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.e{-1:24398} state up:standby seq 1 addr [v2:172.21.15.150:6834/955154875,v1:172.21.15.150:6835/955154875] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T17:02:58.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.622+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2bfc0b2880 msgr2=0x7f2bfc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.622+0000 7f2c12d62700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2bfc0b2880 0x7f2bfc0b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000c00 tx=0x7f2c00005fb0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.622+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 msgr2=0x7f2c0c104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.622+0000 7f2c12d62700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c104a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b530 tx=0x7f2bf800b8f0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 shutdown_connections 2024-05-14T17:02:58.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2bfc0b2880 0x7f2bfc0b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c104040 0x7f2c0c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c0c1024a0 0x7f2c0c1063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 --2- 172.21.15.99:0/1684239481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c0c1036a0 0x7f2c0c104a80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 >> 172.21.15.99:0/1684239481 conn(0x7f2c0c0fda40 msgr2=0x7f2c0c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:58.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 shutdown_connections 2024-05-14T17:02:58.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:02:57.623+0000 7f2c12d62700 1 -- 172.21.15.99:0/1684239481 wait complete. 2024-05-14T17:02:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:02:58.864 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:58 smithi099 ceph-mon[31851]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:58.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:02:58 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1684239481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T17:02:58.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-14T17:02:58.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:58 smithi193 ceph-mon[35962]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:58.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:02:58 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1684239481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T17:02:58.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:58 smithi150 ceph-mon[36204]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:02:58.917 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:02:58 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1684239481' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T17:02:59.167 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:00.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:00 smithi150 ceph-mon[36204]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:00.474 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:00 smithi099 ceph-mon[31851]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:00.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:00 smithi193 ceph-mon[35962]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:00.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.829+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1520053400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41024a0 msgr2=0x7f73c41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:00.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.829+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1520053400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41024a0 0x7f73c41028c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73b40099c0 tx=0x7f73b4009cd0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1520053400 shutdown_connections 2024-05-14T17:03:00.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1520053400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73c4104040 0x7f73c410a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1520053400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41024a0 0x7f73c41028c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1520053400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c41036a0 0x7f73c4103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:00.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1520053400 >> 172.21.15.99:0/1520053400 conn(0x7f73c40fda60 msgr2=0x7f73c40ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:00.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1520053400 shutdown_connections 2024-05-14T17:03:00.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1520053400 wait complete. 2024-05-14T17:03:01.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 Processor -- start 2024-05-14T17:03:01.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- start start 2024-05-14T17:03:01.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73c41024a0 0x7f73c4106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 0x7f73c4104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c4104040 0x7f73c4104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73c410e010 con 0x7f73c41036a0 2024-05-14T17:03:01.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73c410de90 con 0x7f73c41024a0 2024-05-14T17:03:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.830+0000 7f73ca6dc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73c410e190 con 0x7f73c4104040 2024-05-14T17:03:01.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 0x7f73c4104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 0x7f73c4104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41300/0 (socket says 172.21.15.99:41300) 2024-05-14T17:03:01.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 -- 172.21.15.99:0/1949065542 learned_addr learned my addr 172.21.15.99:0/1949065542 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:01.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73c3fff700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73c41024a0 0x7f73c4106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73c8c79700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c4104040 0x7f73c4104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 -- 172.21.15.99:0/1949065542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c4104040 msgr2=0x7f73c4104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:01.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c4104040 0x7f73c4104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 -- 172.21.15.99:0/1949065542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73c41024a0 msgr2=0x7f73c4106430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:01.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73c41024a0 0x7f73c4106430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 -- 172.21.15.99:0/1949065542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c4105870 con 0x7f73c41036a0 2024-05-14T17:03:01.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73c8c79700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c4104040 0x7f73c4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:01.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73bbfff700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 0x7f73c4104a80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b530 tx=0x7f73ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:01.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac015020 con 0x7f73c41036a0 2024-05-14T17:03:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73ac011ce0 con 0x7f73c41036a0 2024-05-14T17:03:01.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac010e40 con 0x7f73c41036a0 2024-05-14T17:03:01.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c41a3980 con 0x7f73c41036a0 2024-05-14T17:03:01.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.831+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c41a3f10 con 0x7f73c41036a0 2024-05-14T17:03:01.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.832+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c406a230 con 0x7f73c41036a0 2024-05-14T17:03:01.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.834+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f73ac018020 con 0x7f73c41036a0 2024-05-14T17:03:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.834+0000 7f73c1ffb700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f73b00b2900 0x7f73b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.834+0000 7f73c3fff700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f73b00b2900 0x7f73b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.834+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f73ac0e1f70 con 0x7f73c41036a0 2024-05-14T17:03:01.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.835+0000 7f73c3fff700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f73b00b2900 0x7f73b00b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73b40038c0 tx=0x7f73b400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:00.838+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73ac01f020 con 0x7f73c41036a0 2024-05-14T17:03:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.044+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f73c41a41a0 con 0x7f73c41036a0 2024-05-14T17:03:01.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.045+0000 7f73c1ffb700 1 -- 172.21.15.99:0/1949065542 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5296 (secure 0 0 0) 0x7f73ac0a9ca0 con 0x7f73c41036a0 2024-05-14T17:03:01.411 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 82 2024-05-14T17:03:01.418 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:03:01.425 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-05-14T16:52:09.109385+0000 2024-05-14T17:03:01.434 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2024-05-14T17:01:04.227948+0000 2024-05-14T17:03:01.442 INFO:teuthology.orchestra.run.smithi099.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-14T17:03:01.448 INFO:teuthology.orchestra.run.smithi099.stdout:crush_version 26 2024-05-14T17:03:01.462 INFO:teuthology.orchestra.run.smithi099.stdout:full_ratio 0.95 2024-05-14T17:03:01.478 INFO:teuthology.orchestra.run.smithi099.stdout:backfillfull_ratio 0.9 2024-05-14T17:03:01.485 INFO:teuthology.orchestra.run.smithi099.stdout:nearfull_ratio 0.85 2024-05-14T17:03:01.491 INFO:teuthology.orchestra.run.smithi099.stdout:require_min_compat_client luminous 2024-05-14T17:03:01.499 INFO:teuthology.orchestra.run.smithi099.stdout:min_compat_client luminous 2024-05-14T17:03:01.505 INFO:teuthology.orchestra.run.smithi099.stdout:require_osd_release squid 2024-05-14T17:03:01.512 INFO:teuthology.orchestra.run.smithi099.stdout:stretch_mode_enabled false 2024-05-14T17:03:01.519 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-14T17:03:01.540 INFO:teuthology.orchestra.run.smithi099.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-14T17:03:01.596 INFO:teuthology.orchestra.run.smithi099.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-14T17:03:01.609 INFO:teuthology.orchestra.run.smithi099.stdout:max_osd 12 2024-05-14T17:03:01.623 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6802/3363601004,v1:172.21.15.99:6803/3363601004] [v2:172.21.15.99:6804/3363601004,v1:172.21.15.99:6805/3363601004] exists,up a0db35f1-6897-4148-8abf-3d8f5945234d 2024-05-14T17:03:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/3718481884,v1:172.21.15.150:6801/3718481884] [v2:172.21.15.150:6802/3718481884,v1:172.21.15.150:6803/3718481884] exists,up 9bc6113d-d89d-4515-a997-f001e08ff626 2024-05-14T17:03:01.653 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/2485896294,v1:172.21.15.193:6801/2485896294] [v2:172.21.15.193:6802/2485896294,v1:172.21.15.193:6803/2485896294] exists,up 4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef 2024-05-14T17:03:01.663 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 up in weight 1 up_from 26 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6810/429217969,v1:172.21.15.99:6811/429217969] [v2:172.21.15.99:6812/429217969,v1:172.21.15.99:6813/429217969] exists,up ad8e6b76-114d-4622-8e95-ea570fb85003 2024-05-14T17:03:01.676 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 up in weight 1 up_from 32 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/1354191038,v1:172.21.15.150:6809/1354191038] [v2:172.21.15.150:6810/1354191038,v1:172.21.15.150:6811/1354191038] exists,up cf410b8e-faab-433d-8af0-db09e90b91b9 2024-05-14T17:03:01.692 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2563526327,v1:172.21.15.193:6809/2563526327] [v2:172.21.15.193:6810/2563526327,v1:172.21.15.193:6811/2563526327] exists,up 1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd 2024-05-14T17:03:01.721 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 up in weight 1 up_from 44 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6818/3623238310,v1:172.21.15.99:6819/3623238310] [v2:172.21.15.99:6820/3623238310,v1:172.21.15.99:6821/3623238310] exists,up 4a01d0f1-49fc-4260-92d1-e75e3ec36b14 2024-05-14T17:03:01.734 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 up in weight 1 up_from 49 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/1186953586,v1:172.21.15.150:6817/1186953586] [v2:172.21.15.150:6818/1186953586,v1:172.21.15.150:6819/1186953586] exists,up 77c663e2-65e8-46c2-9363-bf3fecf22dd9 2024-05-14T17:03:01.745 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 up in weight 1 up_from 55 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/2890842755,v1:172.21.15.193:6817/2890842755] [v2:172.21.15.193:6818/2890842755,v1:172.21.15.193:6819/2890842755] exists,up 0e6281da-4f42-4ba0-8033-cac6194b5a55 2024-05-14T17:03:01.761 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 up in weight 1 up_from 61 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6826/3111395354,v1:172.21.15.99:6827/3111395354] [v2:172.21.15.99:6828/3111395354,v1:172.21.15.99:6829/3111395354] exists,up 302e14e8-2084-4de2-947a-3df7b3f8e45e 2024-05-14T17:03:01.774 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2780871308,v1:172.21.15.150:6825/2780871308] [v2:172.21.15.150:6826/2780871308,v1:172.21.15.150:6827/2780871308] exists,up 64fe2c42-8ae1-4465-a821-ff6118d2a262 2024-05-14T17:03:01.786 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/3625177067,v1:172.21.15.193:6825/3625177067] [v2:172.21.15.193:6826/3625177067,v1:172.21.15.193:6827/3625177067] exists,up cc26027a-0163-4ecd-88da-2029accfe161 2024-05-14T17:03:01.801 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.b [8,11] 2024-05-14T17:03:01.815 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.13 [2,11] 2024-05-14T17:03:01.842 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.1b [2,4] 2024-05-14T17:03:01.898 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.0 [4,7,5,9] 2024-05-14T17:03:01.920 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.18 [3,9] 2024-05-14T17:03:01.934 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.1a [5,6] 2024-05-14T17:03:01.946 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.27 [4,8] 2024-05-14T17:03:01.961 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.2c [4,9] 2024-05-14T17:03:01.973 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/575236794 expires 2024-05-15T16:53:02.369520+0000 2024-05-14T17:03:01.988 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/488380564 expires 2024-05-15T16:53:02.369520+0000 2024-05-14T17:03:02.002 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/1504956360 expires 2024-05-15T16:52:36.088176+0000 2024-05-14T17:03:02.013 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/3420859495 expires 2024-05-15T16:52:36.088176+0000 2024-05-14T17:03:02.030 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2756368842 expires 2024-05-15T16:52:36.088176+0000 2024-05-14T17:03:02.042 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/1504956360 expires 2024-05-15T16:52:36.088176+0000 2024-05-14T17:03:02.054 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/1816772983 expires 2024-05-15T16:53:02.369520+0000 2024-05-14T17:03:02.069 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/3450933746 expires 2024-05-15T16:52:36.088176+0000 2024-05-14T17:03:02.083 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/1463965565 expires 2024-05-15T16:53:02.369520+0000 2024-05-14T17:03:02.097 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/1816772983 expires 2024-05-15T16:53:02.369520+0000 2024-05-14T17:03:02.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f73b00b2900 msgr2=0x7f73b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:02.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f73b00b2900 0x7f73b00b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73b40038c0 tx=0x7f73b400b540 comp rx=0 tx=0).stop 2024-05-14T17:03:02.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 msgr2=0x7f73c4104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:02.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 0x7f73c4104a80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b530 tx=0x7f73ac00b840 comp rx=0 tx=0).stop 2024-05-14T17:03:02.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 shutdown_connections 2024-05-14T17:03:02.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f73b00b2900 0x7f73b00b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c4104040 0x7f73c4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73c41024a0 0x7f73c4106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 --2- 172.21.15.99:0/1949065542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c41036a0 0x7f73c4104a80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 >> 172.21.15.99:0/1949065542 conn(0x7f73c40fda60 msgr2=0x7f73c40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:02.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 shutdown_connections 2024-05-14T17:03:02.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:01.047+0000 7f73ca6dc700 1 -- 172.21.15.99:0/1949065542 wait complete. 2024-05-14T17:03:02.339 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:01 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1949065542' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T17:03:02.391 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-14T17:03:02.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:03:02.426 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:01 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1949065542' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T17:03:02.436 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:01 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1949065542' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T17:03:02.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:02 smithi150 ceph-mon[36204]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:02.482 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:02 smithi099 ceph-mon[31851]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:02 smithi193 ceph-mon[35962]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:04.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 -- 172.21.15.99:0/1593204559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced4108910 msgr2=0x7fced4110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 --2- 172.21.15.99:0/1593204559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced4108910 0x7fced4110490 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00b0d0 tx=0x7fcecc00b3e0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 -- 172.21.15.99:0/1593204559 shutdown_connections 2024-05-14T17:03:04.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 --2- 172.21.15.99:0/1593204559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced4108910 0x7fced4110490 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 --2- 172.21.15.99:0/1593204559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced4103820 0x7fced4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 --2- 172.21.15.99:0/1593204559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced41041d0 0x7fced41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 -- 172.21.15.99:0/1593204559 >> 172.21.15.99:0/1593204559 conn(0x7fced407a6b0 msgr2=0x7fced407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:04.491 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:04 smithi099 ceph-mon[31851]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:04.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 -- 172.21.15.99:0/1593204559 shutdown_connections 2024-05-14T17:03:04.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.387+0000 7fced89b0700 1 -- 172.21.15.99:0/1593204559 wait complete. 2024-05-14T17:03:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 Processor -- start 2024-05-14T17:03:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 -- start start 2024-05-14T17:03:04.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced4103820 0x7fced419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:04.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 0x7fced419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced4108910 0x7fced4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:04.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fced41136e0 con 0x7fced41041d0 2024-05-14T17:03:04.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fced4113560 con 0x7fced4103820 2024-05-14T17:03:04.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.388+0000 7fced89b0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fced4113860 con 0x7fced4108910 2024-05-14T17:03:04.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 0x7fced419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:04.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 0x7fced419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41320/0 (socket says 172.21.15.99:41320) 2024-05-14T17:03:04.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 -- 172.21.15.99:0/1700186410 learned_addr learned my addr 172.21.15.99:0/1700186410 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced259c700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced4103820 0x7fced419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:04.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced2d9d700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced4108910 0x7fced4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 -- 172.21.15.99:0/1700186410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced4108910 msgr2=0x7fced4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced4108910 0x7fced4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 -- 172.21.15.99:0/1700186410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced4103820 msgr2=0x7fced419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced4103820 0x7fced419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 -- 172.21.15.99:0/1700186410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced419a060 con 0x7fced41041d0 2024-05-14T17:03:04.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced1d9b700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 0x7fced419fc30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcec800c9b0 tx=0x7fcec800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:04.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8007b00 con 0x7fced41041d0 2024-05-14T17:03:04.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec800ce50 con 0x7fced41041d0 2024-05-14T17:03:04.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced419a350 con 0x7fced41041d0 2024-05-14T17:03:04.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8018660 con 0x7fced41041d0 2024-05-14T17:03:04.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.389+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced4075770 con 0x7fced41041d0 2024-05-14T17:03:04.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.390+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced406a230 con 0x7fced41041d0 2024-05-14T17:03:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.393+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcec800f450 con 0x7fced41041d0 2024-05-14T17:03:04.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.393+0000 7fcec37fe700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcebc0b2880 0x7fcebc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:04.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.393+0000 7fced259c700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcebc0b2880 0x7fcebc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:05.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.393+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7fcec80e2400 con 0x7fced41041d0 2024-05-14T17:03:05.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.394+0000 7fced259c700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcebc0b2880 0x7fcebc0b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fced4108400 tx=0x7fcec400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.397+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcec8014030 con 0x7fced41041d0 2024-05-14T17:03:05.098 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:04 smithi193 ceph-mon[35962]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:05.112 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:04 smithi150 ceph-mon[36204]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.647+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fced419ae10 con 0x7fced41041d0 2024-05-14T17:03:05.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.647+0000 7fcec37fe700 1 -- 172.21.15.99:0/1700186410 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7fced419ae10 con 0x7fced41041d0 2024-05-14T17:03:05.151 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:03:05.165 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:00:30.962944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:03:05.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcebc0b2880 msgr2=0x7fcebc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:05.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcebc0b2880 0x7fcebc0b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fced4108400 tx=0x7fcec400b410 comp rx=0 tx=0).stop 2024-05-14T17:03:05.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 msgr2=0x7fced419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:05.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 0x7fced419fc30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcec800c9b0 tx=0x7fcec800ccc0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 shutdown_connections 2024-05-14T17:03:05.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7fcebc0b2880 0x7fcebc0b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced4108910 0x7fced4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced4103820 0x7fced419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 --2- 172.21.15.99:0/1700186410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced41041d0 0x7fced419fc30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 >> 172.21.15.99:0/1700186410 conn(0x7fced407a6b0 msgr2=0x7fced41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:05.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 shutdown_connections 2024-05-14T17:03:05.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:04.650+0000 7fced89b0700 1 -- 172.21.15.99:0/1700186410 wait complete. 2024-05-14T17:03:05.470 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:03:05.483 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:05 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1700186410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:05.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:05 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1700186410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:05.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:03:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:05 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1700186410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:05.793 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:06.166 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:06 smithi099 ceph-mon[31851]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:06.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:06 smithi150 ceph-mon[36204]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:06 smithi193 ceph-mon[35962]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:07.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/1977334991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff0101cf0 msgr2=0x7f9ff01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:07.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/1977334991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff0101cf0 0x7f9ff01020d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b0d0 tx=0x7f9fec00b3e0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/1977334991 shutdown_connections 2024-05-14T17:03:07.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/1977334991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ff0108930 0x7f9ff01104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/1977334991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff0101cf0 0x7f9ff01020d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/1977334991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff01026a0 0x7f9ff0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/1977334991 >> 172.21.15.99:0/1977334991 conn(0x7f9ff0078a40 msgr2=0x7f9ff00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:07.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/1977334991 shutdown_connections 2024-05-14T17:03:07.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.420+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/1977334991 wait complete. 2024-05-14T17:03:07.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 Processor -- start 2024-05-14T17:03:07.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 -- start start 2024-05-14T17:03:07.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0101cf0 0x7f9ff019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 0x7f9ff019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ff0108930 0x7f9ff0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113790 con 0x7f9ff01026a0 2024-05-14T17:03:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113610 con 0x7f9ff0101cf0 2024-05-14T17:03:07.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff6c0c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113910 con 0x7f9ff0108930 2024-05-14T17:03:07.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 0x7f9ff019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 0x7f9ff019fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48802/0 (socket says 172.21.15.99:48802) 2024-05-14T17:03:07.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 -- 172.21.15.99:0/3281251980 learned_addr learned my addr 172.21.15.99:0/3281251980 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:07.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff49a8700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0101cf0 0x7f9ff019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 -- 172.21.15.99:0/3281251980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ff0108930 msgr2=0x7f9ff0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:07.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ff0108930 0x7f9ff0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 -- 172.21.15.99:0/3281251980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0101cf0 msgr2=0x7f9ff019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:07.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0101cf0 0x7f9ff019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 -- 172.21.15.99:0/3281251980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019a250 con 0x7f9ff01026a0 2024-05-14T17:03:07.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9ff49a8700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0101cf0 0x7f9ff019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:07.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.421+0000 7f9fe7fff700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 0x7f9ff019fe70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b530 tx=0x7f9fe000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:07.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.422+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0015020 con 0x7f9ff01026a0 2024-05-14T17:03:07.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.422+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe0011ce0 con 0x7f9ff01026a0 2024-05-14T17:03:08.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.422+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019a540 con 0x7f9ff01026a0 2024-05-14T17:03:08.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.422+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0010d20 con 0x7f9ff01026a0 2024-05-14T17:03:08.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.422+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a6800 con 0x7f9ff01026a0 2024-05-14T17:03:08.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.423+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff010bb70 con 0x7f9ff01026a0 2024-05-14T17:03:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.424+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9fe0018020 con 0x7f9ff01026a0 2024-05-14T17:03:08.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.425+0000 7f9fe5ffb700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9fd80b2880 0x7f9fd80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:08.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.425+0000 7f9ff49a8700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9fd80b2880 0x7f9fd80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:08.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.425+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f9fe00e1a40 con 0x7f9ff01026a0 2024-05-14T17:03:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.425+0000 7f9ff49a8700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9fd80b2880 0x7f9fd80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00d010 tx=0x7f9fec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:08.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.430+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fe001f020 con 0x7f9ff01026a0 2024-05-14T17:03:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.679+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ff006a230 con 0x7f9ff01026a0 2024-05-14T17:03:08.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.680+0000 7f9fe5ffb700 1 -- 172.21.15.99:0/3281251980 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7f9fe00a9770 con 0x7f9ff01026a0 2024-05-14T17:03:08.211 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:03:08.223 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:00:30.962944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:03:08.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.682+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9fd80b2880 msgr2=0x7f9fd80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:08.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9fd80b2880 0x7f9fd80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00d010 tx=0x7f9fec00b040 comp rx=0 tx=0).stop 2024-05-14T17:03:08.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 msgr2=0x7f9ff019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:08.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 0x7f9ff019fe70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b530 tx=0x7f9fe000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 shutdown_connections 2024-05-14T17:03:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f9fd80b2880 0x7f9fd80b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ff0108930 0x7f9ff0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0101cf0 0x7f9ff019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 --2- 172.21.15.99:0/3281251980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ff01026a0 0x7f9ff019fe70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 >> 172.21.15.99:0/3281251980 conn(0x7f9ff0078a40 msgr2=0x7f9ff00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:08.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 shutdown_connections 2024-05-14T17:03:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:07.683+0000 7f9ff6c0c700 1 -- 172.21.15.99:0/3281251980 wait complete. 2024-05-14T17:03:08.583 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-05-14T17:03:08.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:08 smithi193 ceph-mon[35962]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:08.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:08 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3281251980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:08.626 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:08 smithi150 ceph-mon[36204]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:08.639 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:08 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3281251980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:08.651 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:08 smithi099 ceph-mon[31851]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:08.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:08 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3281251980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:03:08.674 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:03:08.930 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:09.393 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:09 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:09 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:09.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:09 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:10.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:10 smithi099 ceph-mon[31851]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:10.372 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:10.380 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:10 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:10.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:10 smithi150 ceph-mon[36204]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:10.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:10.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:10 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:10.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:10 smithi193 ceph-mon[35962]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:10.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:10.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:10 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:10.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 -- 172.21.15.99:0/1765228570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c102e00 msgr2=0x7ff29c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:10.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/1765228570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c102e00 0x7ff29c10f2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b0a0 tx=0x7ff29400b3b0 comp rx=0 tx=0).stop 2024-05-14T17:03:10.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 -- 172.21.15.99:0/1765228570 shutdown_connections 2024-05-14T17:03:10.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/1765228570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c102e00 0x7ff29c10f2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:10.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/1765228570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff29c108520 0x7ff29c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/1765228570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29c102440 0x7ff29c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:10.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 -- 172.21.15.99:0/1765228570 >> 172.21.15.99:0/1765228570 conn(0x7ff29c0793f0 msgr2=0x7ff29c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:10.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 -- 172.21.15.99:0/1765228570 shutdown_connections 2024-05-14T17:03:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 -- 172.21.15.99:0/1765228570 wait complete. 2024-05-14T17:03:10.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.624+0000 7ff2a1c92700 1 Processor -- start 2024-05-14T17:03:10.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 -- start start 2024-05-14T17:03:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29c102440 0x7ff29c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:10.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff29c102e00 0x7ff29c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:10.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 0x7ff29c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:10.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1125f0 con 0x7ff29c108520 2024-05-14T17:03:10.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c112470 con 0x7ff29c102e00 2024-05-14T17:03:10.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff2a1c92700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c112770 con 0x7ff29c102440 2024-05-14T17:03:10.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 0x7ff29c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:10.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 0x7ff29c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48816/0 (socket says 172.21.15.99:48816) 2024-05-14T17:03:10.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 -- 172.21.15.99:0/2396402340 learned_addr learned my addr 172.21.15.99:0/2396402340 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29b7fe700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29c102440 0x7ff29c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29affd700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff29c102e00 0x7ff29c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 -- 172.21.15.99:0/2396402340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29c102440 msgr2=0x7ff29c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:11.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29c102440 0x7ff29c19bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 -- 172.21.15.99:0/2396402340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff29c102e00 msgr2=0x7ff29c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff29c102e00 0x7ff29c19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 -- 172.21.15.99:0/2396402340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c1a0d60 con 0x7ff29c108520 2024-05-14T17:03:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.625+0000 7ff29bfff700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 0x7ff29c1a0620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b830 tx=0x7ff294004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:11.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.626+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294021070 con 0x7ff29c108520 2024-05-14T17:03:11.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.626+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff294007e30 con 0x7ff29c108520 2024-05-14T17:03:11.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.626+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c1a0ff0 con 0x7ff29c108520 2024-05-14T17:03:11.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.626+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940047d0 con 0x7ff29c108520 2024-05-14T17:03:11.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.626+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c1a88d0 con 0x7ff29c108520 2024-05-14T17:03:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.627+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c06a230 con 0x7ff29c108520 2024-05-14T17:03:11.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.628+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff294007980 con 0x7ff29c108520 2024-05-14T17:03:11.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.628+0000 7ff298ff9700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff2840b2900 0x7ff2840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:11.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.629+0000 7ff29b7fe700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff2840b2900 0x7ff2840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:11.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.629+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7ff2940e31d0 con 0x7ff29c108520 2024-05-14T17:03:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.629+0000 7ff29b7fe700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff2840b2900 0x7ff2840b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00abd0 tx=0x7ff28c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.632+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2940aaf90 con 0x7ff29c108520 2024-05-14T17:03:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.838+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff29c1a8aa0 con 0x7ff29c108520 2024-05-14T17:03:11.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.840+0000 7ff298ff9700 1 -- 172.21.15.99:0/2396402340 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22695 (secure 0 0 0) 0x7ff2940aa890 con 0x7ff29c108520 2024-05-14T17:03:11.497 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:03:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:01:04.227948+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:03:11.519 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:11.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:11.539 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:11 smithi150 ceph-mon[36204]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:11.550 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:11 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2396402340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:03:11.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff2840b2900 msgr2=0x7ff2840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:11.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff2840b2900 0x7ff2840b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00abd0 tx=0x7ff28c00a380 comp rx=0 tx=0).stop 2024-05-14T17:03:11.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 msgr2=0x7ff29c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:11.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 0x7ff29c1a0620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b830 tx=0x7ff294004ce0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 shutdown_connections 2024-05-14T17:03:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7ff2840b2900 0x7ff2840b4dc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff29c108520 0x7ff29c1a0620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff29c102440 0x7ff29c19bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 --2- 172.21.15.99:0/2396402340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff29c102e00 0x7ff29c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:11.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 >> 172.21.15.99:0/2396402340 conn(0x7ff29c0793f0 msgr2=0x7ff29c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:11.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 shutdown_connections 2024-05-14T17:03:11.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:10.842+0000 7ff2a1c92700 1 -- 172.21.15.99:0/2396402340 wait complete. 2024-05-14T17:03:11.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:11.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:11.772 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:11 smithi099 ceph-mon[31851]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:11.780 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:11 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2396402340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:03:11.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T17:00:11.851265+0000', 'modified': '2024-05-14T17:00:30.962944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24463, 'mds_1': 24448, 'mds_2': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/1891176204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 1891176204}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1891176204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3825469703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3825469703}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3825469703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/3113222034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 3113222034}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 3113222034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.150:6839/3105942045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 3105942045}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 3105942045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.193:6837/4135945134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 4135945134}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 4135945134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/4034311425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 4034311425}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 4034311425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24463, 'qdb_cluster': [24448, 24463, 24443]} 2024-05-14T17:03:11.801 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T17:03:11.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T17:03:11.825 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:11.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:11.842 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:11 smithi193 ceph-mon[35962]: from='mgr.14152 172.21.15.99:0/2891922695' entity='mgr.x' 2024-05-14T17:03:11.850 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:11 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2396402340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:03:12.103 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:12.386 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:12 smithi099 ceph-mon[31851]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:12.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:12 smithi150 ceph-mon[36204]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:12.513 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:12 smithi193 ceph-mon[35962]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:13.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.906+0000 7f05e4a6a700 1 -- 172.21.15.99:0/2084055976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 msgr2=0x7f05e0102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:13.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.906+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/2084055976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e0102950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009990 tx=0x7f05d4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:03:13.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 -- 172.21.15.99:0/2084055976 shutdown_connections 2024-05-14T17:03:13.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/2084055976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e0102e90 0x7f05e010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:13.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/2084055976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05e01084d0 0x7f05e01088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:13.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/2084055976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e0102950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:13.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 -- 172.21.15.99:0/2084055976 >> 172.21.15.99:0/2084055976 conn(0x7f05e0078490 msgr2=0x7f05e00788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:13.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 -- 172.21.15.99:0/2084055976 shutdown_connections 2024-05-14T17:03:13.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 -- 172.21.15.99:0/2084055976 wait complete. 2024-05-14T17:03:14.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 Processor -- start 2024-05-14T17:03:14.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.907+0000 7f05e4a6a700 1 -- start start 2024-05-14T17:03:14.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e0102e90 0x7f05e0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05e01084d0 0x7f05e0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05e01126e0 con 0x7f05e01024d0 2024-05-14T17:03:14.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0112560 con 0x7f05e01084d0 2024-05-14T17:03:14.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05e0112860 con 0x7f05e0102e90 2024-05-14T17:03:14.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48838/0 (socket says 172.21.15.99:48838) 2024-05-14T17:03:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 -- 172.21.15.99:0/1946277080 learned_addr learned my addr 172.21.15.99:0/1946277080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:14.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e0102e90 0x7f05e0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05ded9d700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05e01084d0 0x7f05e0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 -- 172.21.15.99:0/1946277080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e0102e90 msgr2=0x7f05e0198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:14.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e0102e90 0x7f05e0198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:14.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 -- 172.21.15.99:0/1946277080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05e01084d0 msgr2=0x7f05e0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:14.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05e01084d0 0x7f05e0199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:14.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 -- 172.21.15.99:0/1946277080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e01a8220 con 0x7f05e01024d0 2024-05-14T17:03:14.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05de59c700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e01989b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f05d000b530 tx=0x7f05d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:14.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05cf7fe700 1 -- 172.21.15.99:0/1946277080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0015020 con 0x7f05e01024d0 2024-05-14T17:03:14.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05cf7fe700 1 -- 172.21.15.99:0/1946277080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d0011ce0 con 0x7f05e01024d0 2024-05-14T17:03:14.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05cf7fe700 1 -- 172.21.15.99:0/1946277080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0010e40 con 0x7f05e01024d0 2024-05-14T17:03:14.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e01a8510 con 0x7f05e01024d0 2024-05-14T17:03:14.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.908+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e01a8a20 con 0x7f05e01024d0 2024-05-14T17:03:14.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.911+0000 7f05cf7fe700 1 -- 172.21.15.99:0/1946277080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f05d0018020 con 0x7f05e01024d0 2024-05-14T17:03:14.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.912+0000 7f05cf7fe700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f05c80b2880 0x7f05c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.912+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f05c80b2880 0x7f05c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.912+0000 7f05cf7fe700 1 -- 172.21.15.99:0/1946277080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f05d00e1a90 con 0x7f05e01024d0 2024-05-14T17:03:14.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.912+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f05c80b2880 0x7f05c80b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f05d400b5c0 tx=0x7f05d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:14.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.912+0000 7f05ef300b80 1 Processor -- start 2024-05-14T17:03:14.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 -- start start 2024-05-14T17:03:14.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2ed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2ed910 0x55a0ab2edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:14.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab247c40 con 0x55a0ab2e9c10 2024-05-14T17:03:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab2377e0 con 0x55a0ab2e92f0 2024-05-14T17:03:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ef300b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab23b820 con 0x55a0ab2ed910 2024-05-14T17:03:14.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2ed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2ed3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48846/0 (socket says 172.21.15.99:48846) 2024-05-14T17:03:14.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 -- 172.21.15.99:0/700501628 learned_addr learned my addr 172.21.15.99:0/700501628 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05de59c700 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ded9d700 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2ed910 0x55a0ab2edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:14.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 -- 172.21.15.99:0/700501628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2ed910 msgr2=0x55a0ab2edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:14.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2ed910 0x55a0ab2edd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 -- 172.21.15.99:0/700501628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2e92f0 msgr2=0x55a0ab2e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:14.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2e96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 -- 172.21.15.99:0/700501628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ab187880 con 0x55a0ab2e9c10 2024-05-14T17:03:14.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ded9d700 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2ed910 0x55a0ab2edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:14.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.913+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2ed3d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05d4015000 tx=0x7f05d401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad47dec3de03f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:14.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ccff9700 1 -- 172.21.15.99:0/700501628 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4034070 con 0x55a0ab2e9c10 2024-05-14T17:03:14.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ccff9700 1 -- 172.21.15.99:0/700501628 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d4024930 con 0x55a0ab2e9c10 2024-05-14T17:03:14.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ccff9700 1 -- 172.21.15.99:0/700501628 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d402faa0 con 0x55a0ab2e9c10 2024-05-14T17:03:14.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 -- 172.21.15.99:0/700501628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 msgr2=0x55a0ab2ed3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:14.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2ed3d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05d4015000 tx=0x7f05d401f000 comp rx=0 tx=0).stop 2024-05-14T17:03:14.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 -- 172.21.15.99:0/700501628 shutdown_connections 2024-05-14T17:03:14.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2ed910 0x55a0ab2edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:14.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 --2- 172.21.15.99:0/700501628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2ed3d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.914+0000 7f05ef300b80 1 -- 172.21.15.99:0/700501628 >> 172.21.15.99:0/700501628 conn(0x55a0ab214200 msgr2=0x55a0ab2e3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:15.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 -- 172.21.15.99:0/700501628 shutdown_connections 2024-05-14T17:03:15.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 -- 172.21.15.99:0/700501628 wait complete. 2024-05-14T17:03:15.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 -- start start 2024-05-14T17:03:15.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 Processor -- start 2024-05-14T17:03:15.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:15.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2f2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:15.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:15.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab13df50 con 0x55a0ab2e9c10 2024-05-14T17:03:15.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab1d9770 con 0x55a0ab2ed910 2024-05-14T17:03:15.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab2f5b10 con 0x55a0ab2e92f0 2024-05-14T17:03:15.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ef300b80 10 client.0 _ll_register_callbacks cb 0x55a0ab1ce530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:15.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48852/0 (socket says 172.21.15.99:48852) 2024-05-14T17:03:15.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ddd9b700 1 -- 172.21.15.99:0/1647188871 learned_addr learned my addr 172.21.15.99:0/1647188871 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:15.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.915+0000 7f05ded9d700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05de59c700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:15.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ddd9b700 1 -- 172.21.15.99:0/1647188871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 msgr2=0x55a0ab2f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:15.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ddd9b700 1 -- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 msgr2=0x55a0ab2f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:15.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:15.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ddd9b700 1 -- 172.21.15.99:0/1647188871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ab1d6640 con 0x55a0ab2e9c10 2024-05-14T17:03:15.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2f2170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f05d4032040 tx=0x7f05d4024d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbda6346adf12778 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:15.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:03:15.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05c5ffb700 1 -- 172.21.15.99:0/1647188871 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4034040 con 0x55a0ab2e9c10 2024-05-14T17:03:15.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05c5ffb700 1 -- 172.21.15.99:0/1647188871 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f05d402f640 con 0x55a0ab2e9c10 2024-05-14T17:03:15.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d4024930 con 0x55a0ab2e9c10 2024-05-14T17:03:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05c5ffb700 1 -- 172.21.15.99:0/1647188871 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05d402f840 con 0x55a0ab2e9c10 2024-05-14T17:03:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ef300b80 10 client.15078 fetch_fsmap learned FSMap version 25 2024-05-14T17:03:15.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0ab300180 con 0x55a0ab2e9c10 2024-05-14T17:03:15.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05c5ffb700 1 -- 172.21.15.99:0/1647188871 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d4024930 con 0x55a0ab2e9c10 2024-05-14T17:03:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05c5ffb700 1 -- 172.21.15.99:0/1647188871 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f05d400fa80 con 0x55a0ab2e9c10 2024-05-14T17:03:15.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.916+0000 7f05ef300b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:03:15.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05ef300b80 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T17:03:15.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05ef300b80 20 client.15078 populate_metadata read hostname 'smithi099' 2024-05-14T17:03:15.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab303600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05ef300b80 4 client.15078 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 --> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0ab2b5310 con 0x55a0ab3014d0 2024-05-14T17:03:16.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05de59c700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab303600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05de59c700 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab303600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=802ea60cec76b43 server_cookie=6f8d2e2e7d72df9c in_seq=0 out_seq=0 2024-05-14T17:03:16.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.917+0000 7f05c5ffb700 10 client.15078 ms_handle_connect on v2:172.21.15.150:6838/3105942045 2024-05-14T17:03:16.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05c5ffb700 1 -- 172.21.15.99:0/1647188871 <== mds.2 v2:172.21.15.150:6838/3105942045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0ab2b5310 con 0x55a0ab3014d0 2024-05-14T17:03:16.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05c5ffb700 10 client.15078 handle_command_reply: tid=0 2024-05-14T17:03:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 client.15078 shutdown 2024-05-14T17:03:16.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 msgr2=0x55a0ab2f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2f2170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f05d4032040 tx=0x7f05d4024d10 comp rx=0 tx=0).stop 2024-05-14T17:03:16.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 shutdown_connections 2024-05-14T17:03:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab303600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05c5ffb700 0 client.15078 ms_handle_reset on v2:172.21.15.150:6838/3105942045 2024-05-14T17:03:16.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1647188871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2f2170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 >> 172.21.15.99:0/1647188871 conn(0x55a0ab214200 msgr2=0x55a0ab2eede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:16.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 shutdown_connections 2024-05-14T17:03:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 1 -- 172.21.15.99:0/1647188871 wait complete. 2024-05-14T17:03:16.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.918+0000 7f05ef300b80 20 client.15078 trim_cache size 0 max 16384 2024-05-14T17:03:16.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 Processor -- start 2024-05-14T17:03:16.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 -- start start 2024-05-14T17:03:16.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab301940 con 0x55a0ab2e92f0 2024-05-14T17:03:16.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab302010 con 0x55a0ab2ed910 2024-05-14T17:03:16.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ef300b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab3026e0 con 0x55a0ab2e9c10 2024-05-14T17:03:16.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48860/0 (socket says 172.21.15.99:48860) 2024-05-14T17:03:16.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 -- 172.21.15.99:0/1222711306 learned_addr learned my addr 172.21.15.99:0/1222711306 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:16.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ded9d700 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 -- 172.21.15.99:0/1222711306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e9c10 msgr2=0x55a0ab2eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 -- 172.21.15.99:0/1222711306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 msgr2=0x55a0ab2eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:16.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05de59c700 1 -- 172.21.15.99:0/1222711306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ab2ed740 con 0x55a0ab2e92f0 2024-05-14T17:03:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.939+0000 7f05ded9d700 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:16.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05de59c700 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f1460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f05d00a50e0 tx=0x7f05d001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e4c6e10de6fc40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:16.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05c7fff700 1 -- 172.21.15.99:0/1222711306 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d00b1ec0 con 0x55a0ab2e92f0 2024-05-14T17:03:16.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05c7fff700 1 -- 172.21.15.99:0/1222711306 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d00df650 con 0x55a0ab2e92f0 2024-05-14T17:03:16.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05c7fff700 1 -- 172.21.15.99:0/1222711306 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d00a82c0 con 0x55a0ab2e92f0 2024-05-14T17:03:16.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 -- 172.21.15.99:0/1222711306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 msgr2=0x55a0ab2f1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f1460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f05d00a50e0 tx=0x7f05d001c040 comp rx=0 tx=0).stop 2024-05-14T17:03:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 -- 172.21.15.99:0/1222711306 shutdown_connections 2024-05-14T17:03:16.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab2eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e92f0 0x55a0ab2f1460 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 --2- 172.21.15.99:0/1222711306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e9c10 0x55a0ab2eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 -- 172.21.15.99:0/1222711306 >> 172.21.15.99:0/1222711306 conn(0x55a0ab214200 msgr2=0x55a0ab2e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 -- 172.21.15.99:0/1222711306 shutdown_connections 2024-05-14T17:03:16.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.940+0000 7f05ef300b80 1 -- 172.21.15.99:0/1222711306 wait complete. 2024-05-14T17:03:16.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 -- start start 2024-05-14T17:03:16.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 Processor -- start 2024-05-14T17:03:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab30d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab307a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab307f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:16.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab303480 con 0x55a0ab2e9c10 2024-05-14T17:03:16.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab302db0 con 0x55a0ab2ed910 2024-05-14T17:03:16.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0ab303b50 con 0x55a0ab2e92f0 2024-05-14T17:03:16.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ef300b80 10 client.0 _ll_register_callbacks cb 0x55a0ab1ce530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab307a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab307a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48866/0 (socket says 172.21.15.99:48866) 2024-05-14T17:03:16.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 -- 172.21.15.99:0/3839717804 learned_addr learned my addr 172.21.15.99:0/3839717804 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:16.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05de59c700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab30d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ded9d700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab307f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:16.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 -- 172.21.15.99:0/3839717804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 msgr2=0x55a0ab30d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab30d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:16.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 -- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 msgr2=0x55a0ab307f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:16.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab307f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ddd9b700 1 -- 172.21.15.99:0/3839717804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ab308770 con 0x55a0ab2e9c10 2024-05-14T17:03:17.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.941+0000 7f05ded9d700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab307f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:17.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ddd9b700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab307a30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009750 tx=0x7f05d402f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa192bbb0caa10b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:17.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:03:17.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05c5ffb700 1 -- 172.21.15.99:0/3839717804 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4048070 con 0x55a0ab2e9c10 2024-05-14T17:03:17.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05c5ffb700 1 -- 172.21.15.99:0/3839717804 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f05d403a690 con 0x55a0ab2e9c10 2024-05-14T17:03:17.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d00df650 con 0x55a0ab2e9c10 2024-05-14T17:03:17.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05c5ffb700 1 -- 172.21.15.99:0/3839717804 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05d4034040 con 0x55a0ab2e9c10 2024-05-14T17:03:17.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 10 client.15084 fetch_fsmap learned FSMap version 25 2024-05-14T17:03:17.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0ab36e5a0 con 0x55a0ab2e9c10 2024-05-14T17:03:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05c5ffb700 1 -- 172.21.15.99:0/3839717804 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d00df650 con 0x55a0ab2e9c10 2024-05-14T17:03:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05c5ffb700 1 -- 172.21.15.99:0/3839717804 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f05d404b020 con 0x55a0ab2e9c10 2024-05-14T17:03:17.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:03:17.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T17:03:17.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 20 client.15084 populate_metadata read hostname 'smithi099' 2024-05-14T17:03:17.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab371cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:17.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 4 client.15084 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:03:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.942+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 --> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] -- command(tid 0: {"prefix": "status"}) -- 0x55a0ab249b90 con 0x55a0ab3014d0 2024-05-14T17:03:17.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05de59c700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab371cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:17.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05de59c700 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab371cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f646cc266af28562 server_cookie=a885d060305215cd in_seq=0 out_seq=0 2024-05-14T17:03:17.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05c5ffb700 10 client.15084 ms_handle_connect on v2:172.21.15.150:6838/3105942045 2024-05-14T17:03:17.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05c5ffb700 1 -- 172.21.15.99:0/3839717804 <== mds.2 v2:172.21.15.150:6838/3105942045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a0ab249b90 con 0x55a0ab3014d0 2024-05-14T17:03:17.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05c5ffb700 10 client.15084 handle_command_reply: tid=0 2024-05-14T17:03:17.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 client.15084 shutdown 2024-05-14T17:03:17.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 msgr2=0x55a0ab307a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:17.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab307a30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009750 tx=0x7f05d402f6a0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 shutdown_connections 2024-05-14T17:03:17.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x55a0ab3014d0 0x55a0ab371cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0ab2ed910 0x55a0ab307f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ab2e92f0 0x55a0ab30d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 --2- 172.21.15.99:0/3839717804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a0ab2e9c10 0x55a0ab307a30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 >> 172.21.15.99:0/3839717804 conn(0x55a0ab214200 msgr2=0x55a0ab2ec730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:17.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05c5ffb700 0 client.15084 ms_handle_reset on v2:172.21.15.150:6838/3105942045 2024-05-14T17:03:17.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.943+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 shutdown_connections 2024-05-14T17:03:17.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05ef300b80 1 -- 172.21.15.99:0/3839717804 wait complete. 2024-05-14T17:03:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05ef300b80 20 client.15084 trim_cache size 0 max 16384 2024-05-14T17:03:17.598 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:03:17.610 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:03:17.619 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-05-14T17:03:17.633 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24443, 2024-05-14T17:03:17.648 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:03:17.659 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:03:17.668 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:03:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 167.827494771, 2024-05-14T17:03:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-05-14T17:03:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 82, 2024-05-14T17:03:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T17:03:17.814 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 199.40813172200001 2024-05-14T17:03:17.822 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:03:17.865 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:14 smithi193 ceph-mon[35962]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:17.873 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:16 smithi193 ceph-mon[35962]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:17.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:14 smithi150 ceph-mon[36204]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:17.893 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:16 smithi150 ceph-mon[36204]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:17.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f05c80b2880 msgr2=0x7f05c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:17.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f05c80b2880 0x7f05c80b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f05d400b5c0 tx=0x7f05d4005960 comp rx=0 tx=0).stop 2024-05-14T17:03:17.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 msgr2=0x7f05e01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:17.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e01989b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f05d000b530 tx=0x7f05d000b840 comp rx=0 tx=0).stop 2024-05-14T17:03:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 shutdown_connections 2024-05-14T17:03:17.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f05c80b2880 0x7f05c80b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05e01084d0 0x7f05e0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e01024d0 0x7f05e01989b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 --2- 172.21.15.99:0/1946277080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e0102e90 0x7f05e0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 >> 172.21.15.99:0/1946277080 conn(0x7f05e0078490 msgr2=0x7f05e0100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:18.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 shutdown_connections 2024-05-14T17:03:18.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:13.944+0000 7f05e4a6a700 1 -- 172.21.15.99:0/1946277080 wait complete. 2024-05-14T17:03:18.049 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:14 smithi099 ceph-mon[31851]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:18.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-14T17:03:18.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:16 smithi099 ceph-mon[31851]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:18.373 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:18 smithi150 ceph-mon[36204]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:18.494 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:18 smithi099 ceph-mon[31851]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:18.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:18 smithi193 ceph-mon[35962]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:19.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 -- 172.21.15.99:0/323370445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c006bc80 msgr2=0x7f43c006c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:19.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/323370445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c006bc80 0x7f43c006c130 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43b800b0a0 tx=0x7f43b800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 -- 172.21.15.99:0/323370445 shutdown_connections 2024-05-14T17:03:19.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/323370445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c006bc80 0x7f43c006c130 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/323370445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c0108630 0x7f43c0106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/323370445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43c0106570 0x7f43c01069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 -- 172.21.15.99:0/323370445 >> 172.21.15.99:0/323370445 conn(0x7f43c0079240 msgr2=0x7f43c0079650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:19.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 -- 172.21.15.99:0/323370445 shutdown_connections 2024-05-14T17:03:20.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 -- 172.21.15.99:0/323370445 wait complete. 2024-05-14T17:03:20.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 Processor -- start 2024-05-14T17:03:20.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.888+0000 7f43c7cd4700 1 -- start start 2024-05-14T17:03:20.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c006bc80 0x7f43c01a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43c0106570 0x7f43c01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 0x7f43c01a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0119e70 con 0x7f43c0108630 2024-05-14T17:03:20.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0119cf0 con 0x7f43c006bc80 2024-05-14T17:03:20.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0119ff0 con 0x7f43c0106570 2024-05-14T17:03:20.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 0x7f43c01a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 0x7f43c01a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57700/0 (socket says 172.21.15.99:57700) 2024-05-14T17:03:20.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 -- 172.21.15.99:0/3279145153 learned_addr learned my addr 172.21.15.99:0/3279145153 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:20.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c526f700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43c0106570 0x7f43c01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 -- 172.21.15.99:0/3279145153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43c0106570 msgr2=0x7f43c01a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:20.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43c0106570 0x7f43c01a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 -- 172.21.15.99:0/3279145153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c006bc80 msgr2=0x7f43c01a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:20.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c5a70700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c006bc80 0x7f43c01a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c006bc80 0x7f43c01a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 -- 172.21.15.99:0/3279145153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c01a5250 con 0x7f43c0108630 2024-05-14T17:03:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c5a70700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c006bc80 0x7f43c01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:20.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c6271700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 0x7f43c01a4b10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f43b800b070 tx=0x7f43b800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:20.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43b6ffd700 1 -- 172.21.15.99:0/3279145153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8021020 con 0x7f43c0108630 2024-05-14T17:03:20.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c01a54e0 con 0x7f43c0108630 2024-05-14T17:03:20.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43b6ffd700 1 -- 172.21.15.99:0/3279145153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b8007c80 con 0x7f43c0108630 2024-05-14T17:03:20.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01acdc0 con 0x7f43c0108630 2024-05-14T17:03:20.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.889+0000 7f43b6ffd700 1 -- 172.21.15.99:0/3279145153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b801d8b0 con 0x7f43c0108630 2024-05-14T17:03:20.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.892+0000 7f43b6ffd700 1 -- 172.21.15.99:0/3279145153 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f43b8024020 con 0x7f43c0108630 2024-05-14T17:03:20.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.893+0000 7f43b6ffd700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f43ac0b2880 0x7f43ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.893+0000 7f43c5a70700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f43ac0b2880 0x7f43ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.893+0000 7f43b6ffd700 1 -- 172.21.15.99:0/3279145153 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f43b800f020 con 0x7f43c0108630 2024-05-14T17:03:20.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.893+0000 7f43c5a70700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f43ac0b2880 0x7f43ac0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43c007a6d0 tx=0x7f43b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.893+0000 7f43d256ab80 1 Processor -- start 2024-05-14T17:03:20.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 -- start start 2024-05-14T17:03:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a584380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 0x56355a57e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a57f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56355a4dcd20 con 0x56355a5848c0 2024-05-14T17:03:20.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56355a4cc8c0 con 0x56355a583fa0 2024-05-14T17:03:20.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43d256ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355a4d0900 con 0x56355a57edc0 2024-05-14T17:03:20.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 0x56355a57e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 0x56355a57e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57716/0 (socket says 172.21.15.99:57716) 2024-05-14T17:03:20.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 -- 172.21.15.99:0/3706883422 learned_addr learned my addr 172.21.15.99:0/3706883422 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c5a70700 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a584380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 -- 172.21.15.99:0/3706883422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 msgr2=0x56355a57f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:20.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a57f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 -- 172.21.15.99:0/3706883422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 msgr2=0x56355a584380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a584380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 -- 172.21.15.99:0/3706883422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355a475f60 con 0x56355a5848c0 2024-05-14T17:03:20.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43c526f700 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 0x56355a57e7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00ad10 tx=0x7f43bc00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d7ebc43a1110e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:20.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.894+0000 7f43abfff700 1 -- 172.21.15.99:0/3706883422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00ae60 con 0x56355a5848c0 2024-05-14T17:03:20.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43abfff700 1 -- 172.21.15.99:0/3706883422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43bc003680 con 0x56355a5848c0 2024-05-14T17:03:20.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43abfff700 1 -- 172.21.15.99:0/3706883422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc018690 con 0x56355a5848c0 2024-05-14T17:03:20.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 -- 172.21.15.99:0/3706883422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 msgr2=0x56355a57e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:20.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 0x56355a57e7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00ad10 tx=0x7f43bc00f6a0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 -- 172.21.15.99:0/3706883422 shutdown_connections 2024-05-14T17:03:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a57f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a584380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 --2- 172.21.15.99:0/3706883422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a5848c0 0x56355a57e7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 -- 172.21.15.99:0/3706883422 >> 172.21.15.99:0/3706883422 conn(0x56355a4abd60 msgr2=0x56355a578e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:20.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 -- 172.21.15.99:0/3706883422 shutdown_connections 2024-05-14T17:03:20.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.895+0000 7f43d256ab80 1 -- 172.21.15.99:0/3706883422 wait complete. 2024-05-14T17:03:20.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 -- start start 2024-05-14T17:03:20.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 Processor -- start 2024-05-14T17:03:20.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a57edc0 0x56355a58a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56355a3d2f50 con 0x56355a583fa0 2024-05-14T17:03:21.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56355a434d50 con 0x56355a57edc0 2024-05-14T17:03:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355a58b2d0 con 0x56355a5848c0 2024-05-14T17:03:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43d256ab80 10 client.0 _ll_register_callbacks cb 0x56355a467900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:21.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43c526f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43c526f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57728/0 (socket says 172.21.15.99:57728) 2024-05-14T17:03:21.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.896+0000 7f43c526f700 1 -- 172.21.15.99:0/756930981 learned_addr learned my addr 172.21.15.99:0/756930981 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:21.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c5a70700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a57edc0 0x56355a58a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c6271700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c526f700 1 -- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 msgr2=0x56355a587f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:21.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c526f700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c526f700 1 -- 172.21.15.99:0/756930981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a57edc0 msgr2=0x56355a58a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:21.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c526f700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a57edc0 0x56355a58a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c526f700 1 -- 172.21.15.99:0/756930981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355a41c880 con 0x56355a583fa0 2024-05-14T17:03:21.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c5a70700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a57edc0 0x56355a58a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:21.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43c526f700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5879e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180182b33651fcaa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:21.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:03:21.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43a97fa700 1 -- 172.21.15.99:0/756930981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc026020 con 0x56355a583fa0 2024-05-14T17:03:21.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43a97fa700 1 -- 172.21.15.99:0/756930981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f43bc005450 con 0x56355a583fa0 2024-05-14T17:03:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc003680 con 0x56355a583fa0 2024-05-14T17:03:21.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43a97fa700 1 -- 172.21.15.99:0/756930981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43bc00ae60 con 0x56355a583fa0 2024-05-14T17:03:21.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43d256ab80 10 client.15096 fetch_fsmap learned FSMap version 25 2024-05-14T17:03:21.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56355a486710 con 0x56355a583fa0 2024-05-14T17:03:21.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.897+0000 7f43a97fa700 1 -- 172.21.15.99:0/756930981 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43bc003680 con 0x56355a583fa0 2024-05-14T17:03:21.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43a97fa700 1 -- 172.21.15.99:0/756930981 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f43bc022460 con 0x56355a583fa0 2024-05-14T17:03:21.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43d256ab80 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:03:21.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43d256ab80 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T17:03:21.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43d256ab80 20 client.15096 populate_metadata read hostname 'smithi099' 2024-05-14T17:03:21.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43d256ab80 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a597ab0 0x56355a598f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43d256ab80 4 client.15096 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:21.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 --> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56355a4dec70 con 0x56355a597ab0 2024-05-14T17:03:21.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43c5a70700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a597ab0 0x56355a598f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43a97fa700 10 client.15096 ms_handle_connect on v2:172.21.15.193:6836/4135945134 2024-05-14T17:03:21.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.898+0000 7f43c5a70700 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a597ab0 0x56355a598f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7bf18cdc6f213c0 server_cookie=2f316c496eb39d74 in_seq=0 out_seq=0 2024-05-14T17:03:21.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.899+0000 7f43a97fa700 1 -- 172.21.15.99:0/756930981 <== mds.1 v2:172.21.15.193:6836/4135945134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56355a4dec70 con 0x56355a597ab0 2024-05-14T17:03:21.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.899+0000 7f43a97fa700 10 client.15096 handle_command_reply: tid=0 2024-05-14T17:03:21.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.899+0000 7f43d256ab80 1 client.15096 shutdown 2024-05-14T17:03:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.899+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 msgr2=0x56355a5879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:21.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.899+0000 7f43d256ab80 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5879e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc0042e0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 shutdown_connections 2024-05-14T17:03:21.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a597ab0 0x56355a598f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a57edc0 0x56355a58a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43a97fa700 0 client.15096 ms_handle_reset on v2:172.21.15.193:6836/4135945134 2024-05-14T17:03:21.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 --2- 172.21.15.99:0/756930981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5879e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 >> 172.21.15.99:0/756930981 conn(0x56355a4abd60 msgr2=0x56355a578630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:21.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 shutdown_connections 2024-05-14T17:03:21.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 1 -- 172.21.15.99:0/756930981 wait complete. 2024-05-14T17:03:21.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.900+0000 7f43d256ab80 20 client.15096 trim_cache size 0 max 16384 2024-05-14T17:03:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 Processor -- start 2024-05-14T17:03:21.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 -- start start 2024-05-14T17:03:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 0x56355a57e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a596cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56355a597260 con 0x56355a57edc0 2024-05-14T17:03:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56355a597930 con 0x56355a583fa0 2024-05-14T17:03:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.920+0000 7f43d256ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355a598000 con 0x56355a5848c0 2024-05-14T17:03:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 0x56355a57e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 0x56355a57e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57744/0 (socket says 172.21.15.99:57744) 2024-05-14T17:03:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 -- 172.21.15.99:0/1274571658 learned_addr learned my addr 172.21.15.99:0/1274571658 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c526f700 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a596cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:22.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 -- 172.21.15.99:0/1274571658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 msgr2=0x56355a587ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:22.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 -- 172.21.15.99:0/1274571658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 msgr2=0x56355a596cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:22.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a596cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 -- 172.21.15.99:0/1274571658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355a5845d0 con 0x56355a57edc0 2024-05-14T17:03:22.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c526f700 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a596cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:22.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43c5a70700 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 0x56355a57e7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f43b0007540 tx=0x7f43b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641f4953f392dd3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:22.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43ab7fe700 1 -- 172.21.15.99:0/1274571658 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000cdf0 con 0x56355a57edc0 2024-05-14T17:03:22.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43ab7fe700 1 -- 172.21.15.99:0/1274571658 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b0023cc0 con 0x56355a57edc0 2024-05-14T17:03:22.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.921+0000 7f43ab7fe700 1 -- 172.21.15.99:0/1274571658 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0026910 con 0x56355a57edc0 2024-05-14T17:03:22.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 -- 172.21.15.99:0/1274571658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 msgr2=0x56355a57e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:22.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 0x56355a57e7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f43b0007540 tx=0x7f43b0012000 comp rx=0 tx=0).stop 2024-05-14T17:03:22.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 -- 172.21.15.99:0/1274571658 shutdown_connections 2024-05-14T17:03:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a5848c0 0x56355a587ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a57edc0 0x56355a57e7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1274571658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a583fa0 0x56355a596cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 -- 172.21.15.99:0/1274571658 >> 172.21.15.99:0/1274571658 conn(0x56355a4abd60 msgr2=0x56355a5877c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:22.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 -- 172.21.15.99:0/1274571658 shutdown_connections 2024-05-14T17:03:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 -- 172.21.15.99:0/1274571658 wait complete. 2024-05-14T17:03:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 -- start start 2024-05-14T17:03:22.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.922+0000 7f43d256ab80 1 Processor -- start 2024-05-14T17:03:22.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a578380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:22.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a5848c0 0x56355a578e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:22.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56355a598da0 con 0x56355a583fa0 2024-05-14T17:03:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56355a5986d0 con 0x56355a5848c0 2024-05-14T17:03:22.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56355a599470 con 0x56355a57edc0 2024-05-14T17:03:22.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 10 client.0 _ll_register_callbacks cb 0x56355a467900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:22.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:22.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5788c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57752/0 (socket says 172.21.15.99:57752) 2024-05-14T17:03:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c5a70700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a578380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 -- 172.21.15.99:0/1605661667 learned_addr learned my addr 172.21.15.99:0/1605661667 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:22.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c6271700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a5848c0 0x56355a578e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:22.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 -- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 msgr2=0x56355a578380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:22.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a578380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 -- 172.21.15.99:0/1605661667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a5848c0 msgr2=0x56355a578e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a5848c0 0x56355a578e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 -- 172.21.15.99:0/1605661667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355a5a0800 con 0x56355a583fa0 2024-05-14T17:03:22.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c6271700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a5848c0 0x56355a578e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:22.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43c526f700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5788c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f43bc022800 tx=0x7f43bc00a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3f4a5dcd5d6e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:03:22.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43a97fa700 1 -- 172.21.15.99:0/1605661667 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc026040 con 0x56355a583fa0 2024-05-14T17:03:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43a97fa700 1 -- 172.21.15.99:0/1605661667 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f43bc0229f0 con 0x56355a583fa0 2024-05-14T17:03:22.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b0023cc0 con 0x56355a583fa0 2024-05-14T17:03:22.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.923+0000 7f43a97fa700 1 -- 172.21.15.99:0/1605661667 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43bc004400 con 0x56355a583fa0 2024-05-14T17:03:22.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-05-14T17:03:22.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56355a603230 con 0x56355a583fa0 2024-05-14T17:03:22.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43a97fa700 1 -- 172.21.15.99:0/1605661667 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b0023cc0 con 0x56355a583fa0 2024-05-14T17:03:22.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43a97fa700 1 -- 172.21.15.99:0/1605661667 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f43bc004810 con 0x56355a583fa0 2024-05-14T17:03:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:03:22.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T17:03:22.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 20 client.15102 populate_metadata read hostname 'smithi099' 2024-05-14T17:03:22.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a607e50 0x56355a606b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:22.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 4 client.15102 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:03:22.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 --> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] -- command(tid 0: {"prefix": "status"}) -- 0x56355a4d8450 con 0x56355a607e50 2024-05-14T17:03:22.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.924+0000 7f43c5a70700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a607e50 0x56355a606b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:22.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43a97fa700 10 client.15102 ms_handle_connect on v2:172.21.15.193:6836/4135945134 2024-05-14T17:03:22.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43c5a70700 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a607e50 0x56355a606b60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0d829311b93b3ef server_cookie=729122be62bd4454 in_seq=0 out_seq=0 2024-05-14T17:03:22.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43a97fa700 1 -- 172.21.15.99:0/1605661667 <== mds.1 v2:172.21.15.193:6836/4135945134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56355a4d8450 con 0x56355a607e50 2024-05-14T17:03:22.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43a97fa700 10 client.15102 handle_command_reply: tid=0 2024-05-14T17:03:22.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 client.15102 shutdown 2024-05-14T17:03:23.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 msgr2=0x56355a5788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:23.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5788c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f43bc022800 tx=0x7f43bc00a720 comp rx=0 tx=0).stop 2024-05-14T17:03:23.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 shutdown_connections 2024-05-14T17:03:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x56355a607e50 0x56355a606b60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56355a5848c0 0x56355a578e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56355a57edc0 0x56355a578380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43a97fa700 0 client.15102 ms_handle_reset on v2:172.21.15.193:6836/4135945134 2024-05-14T17:03:23.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 --2- 172.21.15.99:0/1605661667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56355a583fa0 0x56355a5788c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 >> 172.21.15.99:0/1605661667 conn(0x56355a4abd60 msgr2=0x56355a584ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:23.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 shutdown_connections 2024-05-14T17:03:23.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 1 -- 172.21.15.99:0/1605661667 wait complete. 2024-05-14T17:03:23.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.925+0000 7f43d256ab80 20 client.15102 trim_cache size 0 max 16384 2024-05-14T17:03:23.123 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:03:23.134 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:03:23.144 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-05-14T17:03:23.151 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24448, 2024-05-14T17:03:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:03:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:03:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:03:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 174.816064294, 2024-05-14T17:03:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-05-14T17:03:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 82, 2024-05-14T17:03:23.299 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T17:03:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 220.35660549299999 2024-05-14T17:03:23.318 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:03:23.331 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:20 smithi193 ceph-mon[35962]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:23.341 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:22 smithi193 ceph-mon[35962]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:23.352 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:20 smithi150 ceph-mon[36204]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:23.365 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:22 smithi150 ceph-mon[36204]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:23.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f43ac0b2880 msgr2=0x7f43ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f43ac0b2880 0x7f43ac0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43c007a6d0 tx=0x7f43b0008040 comp rx=0 tx=0).stop 2024-05-14T17:03:23.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 msgr2=0x7f43c01a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:23.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 0x7f43c01a4b10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f43b800b070 tx=0x7f43b800ce50 comp rx=0 tx=0).stop 2024-05-14T17:03:23.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 shutdown_connections 2024-05-14T17:03:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f43ac0b2880 0x7f43ac0b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43c0108630 0x7f43c01a4b10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43c006bc80 0x7f43c01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 --2- 172.21.15.99:0/3279145153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43c0106570 0x7f43c01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:23.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 >> 172.21.15.99:0/3279145153 conn(0x7f43c0079240 msgr2=0x7f43c01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:23.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 shutdown_connections 2024-05-14T17:03:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:19.926+0000 7f43c7cd4700 1 -- 172.21.15.99:0/3279145153 wait complete. 2024-05-14T17:03:23.568 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:20 smithi099 ceph-mon[31851]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:23.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T17:03:23.595 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:22 smithi099 ceph-mon[31851]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-14T17:03:23.858 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:24.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:24 smithi099 ceph-mon[31851]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:24.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:24 smithi150 ceph-mon[36204]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:24.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:24 smithi193 ceph-mon[35962]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 -- 172.21.15.99:0/3029473023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 msgr2=0x7f2b6410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:25.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 --2- 172.21.15.99:0/3029473023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6410f2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b0a0 tx=0x7f2b5c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 -- 172.21.15.99:0/3029473023 shutdown_connections 2024-05-14T17:03:25.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 --2- 172.21.15.99:0/3029473023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6410f2c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 --2- 172.21.15.99:0/3029473023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b641073c0 0x7f2b641077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 --2- 172.21.15.99:0/3029473023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b64102470 0x7f2b641028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 -- 172.21.15.99:0/3029473023 >> 172.21.15.99:0/3029473023 conn(0x7f2b640784e0 msgr2=0x7f2b640788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:25.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 -- 172.21.15.99:0/3029473023 shutdown_connections 2024-05-14T17:03:25.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.469+0000 7f2b6b844700 1 -- 172.21.15.99:0/3029473023 wait complete. 2024-05-14T17:03:25.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 Processor -- start 2024-05-14T17:03:25.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 -- start start 2024-05-14T17:03:25.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b64102470 0x7f2b6419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b641073c0 0x7f2b641a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64112690 con 0x7f2b64102e30 2024-05-14T17:03:25.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64112510 con 0x7f2b641073c0 2024-05-14T17:03:25.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b6b844700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64112810 con 0x7f2b64102470 2024-05-14T17:03:25.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:25.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6419b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36596/0 (socket says 172.21.15.99:36596) 2024-05-14T17:03:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b68ddf700 1 -- 172.21.15.99:0/1179037952 learned_addr learned my addr 172.21.15.99:0/1179037952 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:25.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.470+0000 7f2b695e0700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b64102470 0x7f2b6419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:25.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b69de1700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b641073c0 0x7f2b641a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:25.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b68ddf700 1 -- 172.21.15.99:0/1179037952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b64102470 msgr2=0x7f2b6419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:25.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b64102470 0x7f2b6419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b68ddf700 1 -- 172.21.15.99:0/1179037952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b641073c0 msgr2=0x7f2b641a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:25.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b641073c0 0x7f2b641a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:25.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b68ddf700 1 -- 172.21.15.99:0/1179037952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b641a60b0 con 0x7f2b64102e30 2024-05-14T17:03:25.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6419b030 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000b530 tx=0x7f2b6000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:25.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b5a7fc700 1 -- 172.21.15.99:0/1179037952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60015020 con 0x7f2b64102e30 2024-05-14T17:03:25.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641a62e0 con 0x7f2b64102e30 2024-05-14T17:03:25.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b5a7fc700 1 -- 172.21.15.99:0/1179037952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b60011ce0 con 0x7f2b64102e30 2024-05-14T17:03:25.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b5a7fc700 1 -- 172.21.15.99:0/1179037952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60010d20 con 0x7f2b64102e30 2024-05-14T17:03:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.471+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a6840 con 0x7f2b64102e30 2024-05-14T17:03:25.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.473+0000 7f2b5a7fc700 1 -- 172.21.15.99:0/1179037952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2b60018020 con 0x7f2b64102e30 2024-05-14T17:03:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.474+0000 7f2b5a7fc700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2b500b2880 0x7f2b500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:25.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.474+0000 7f2b695e0700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2b500b2880 0x7f2b500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:25.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.474+0000 7f2b5a7fc700 1 -- 172.21.15.99:0/1179037952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f2b600e1a70 con 0x7f2b64102e30 2024-05-14T17:03:25.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.474+0000 7f2b695e0700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2b500b2880 0x7f2b500b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005950 tx=0x7f2b540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:25.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 Processor -- start 2024-05-14T17:03:25.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 -- start start 2024-05-14T17:03:26.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686380 0x55a861686760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 0x55a861680bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a861681630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8615dad20 con 0x55a861686ca0 2024-05-14T17:03:26.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a8615ca8c0 con 0x55a8616811d0 2024-05-14T17:03:26.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b760dab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a8615ce900 con 0x55a861686380 2024-05-14T17:03:26.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 0x55a861680bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 0x55a861680bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36600/0 (socket says 172.21.15.99:36600) 2024-05-14T17:03:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3088792574 learned_addr learned my addr 172.21.15.99:0/3088792574 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:26.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b69de1700 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a861681630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.475+0000 7f2b695e0700 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686380 0x55a861686760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3088792574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686380 msgr2=0x55a861686760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:26.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686380 0x55a861686760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3088792574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 msgr2=0x55a861681630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:26.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a861681630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3088792574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861573f60 con 0x55a861686ca0 2024-05-14T17:03:26.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 0x55a861680bb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2b600a62b0 tx=0x7f2b60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4dee51237bff64 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:26.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b4f7fe700 1 -- 172.21.15.99:0/3088792574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600a9040 con 0x55a861686ca0 2024-05-14T17:03:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b4f7fe700 1 -- 172.21.15.99:0/3088792574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600ad3f0 con 0x55a861686ca0 2024-05-14T17:03:26.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.476+0000 7f2b4f7fe700 1 -- 172.21.15.99:0/3088792574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b1c10 con 0x55a861686ca0 2024-05-14T17:03:26.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 -- 172.21.15.99:0/3088792574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 msgr2=0x55a861680bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 0x55a861680bb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2b600a62b0 tx=0x7f2b60011000 comp rx=0 tx=0).stop 2024-05-14T17:03:26.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 -- 172.21.15.99:0/3088792574 shutdown_connections 2024-05-14T17:03:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a861681630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686380 0x55a861686760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3088792574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686ca0 0x55a861680bb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 -- 172.21.15.99:0/3088792574 >> 172.21.15.99:0/3088792574 conn(0x55a8615a9d60 msgr2=0x55a861676e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:26.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 -- 172.21.15.99:0/3088792574 shutdown_connections 2024-05-14T17:03:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 -- 172.21.15.99:0/3088792574 wait complete. 2024-05-14T17:03:26.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 -- start start 2024-05-14T17:03:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.477+0000 7f2b760dab80 1 Processor -- start 2024-05-14T17:03:26.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 0x55a86168ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686380 0x55a861689e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a86168a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8614d0f50 con 0x55a8616811d0 2024-05-14T17:03:26.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a861532d50 con 0x55a861686380 2024-05-14T17:03:26.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a86168d6a0 con 0x55a861686ca0 2024-05-14T17:03:26.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 10 client.0 _ll_register_callbacks cb 0x55a861565900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 0x55a86168ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 0x55a86168ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36606/0 (socket says 172.21.15.99:36606) 2024-05-14T17:03:26.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 -- 172.21.15.99:0/3253100134 learned_addr learned my addr 172.21.15.99:0/3253100134 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:26.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b69de1700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a86168a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686380 0x55a861689e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 -- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 msgr2=0x55a86168a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a86168a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 -- 172.21.15.99:0/3253100134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686380 msgr2=0x55a861689e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:26.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686380 0x55a861689e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:26.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 -- 172.21.15.99:0/3253100134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86151a880 con 0x55a8616811d0 2024-05-14T17:03:26.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b69de1700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a86168a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:26.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686380 0x55a861689e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:26.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b695e0700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 0x55a86168ca70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b540 tx=0x7f2b54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c3b47c0def63f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:26.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:03:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b4cff9700 1 -- 172.21.15.99:0/3253100134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401f430 con 0x55a8616811d0 2024-05-14T17:03:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b4cff9700 1 -- 172.21.15.99:0/3253100134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f2b5401f5d0 con 0x55a8616811d0 2024-05-14T17:03:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b600ad3f0 con 0x55a8616811d0 2024-05-14T17:03:26.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b4cff9700 1 -- 172.21.15.99:0/3253100134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b5401f7d0 con 0x55a8616811d0 2024-05-14T17:03:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.478+0000 7f2b760dab80 10 client.15114 fetch_fsmap learned FSMap version 25 2024-05-14T17:03:26.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a861584710 con 0x55a8616811d0 2024-05-14T17:03:26.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b4cff9700 1 -- 172.21.15.99:0/3253100134 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600ad3f0 con 0x55a8616811d0 2024-05-14T17:03:26.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b4cff9700 1 -- 172.21.15.99:0/3253100134 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2b54027b30 con 0x55a8616811d0 2024-05-14T17:03:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:03:26.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T17:03:26.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 20 client.15114 populate_metadata read hostname 'smithi099' 2024-05-14T17:03:26.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86169a2f0 0x55a86169b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 4 client.15114 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:26.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8615dcc70 con 0x55a86169a2f0 2024-05-14T17:03:27.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.479+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86169a2f0 0x55a86169b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:27.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.480+0000 7f2b4cff9700 10 client.15114 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:03:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.480+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86169a2f0 0x55a86169b760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1216443e9f8dc9a server_cookie=2629365e1ebc7b9d in_seq=0 out_seq=0 2024-05-14T17:03:27.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.480+0000 7f2b4cff9700 1 -- 172.21.15.99:0/3253100134 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8615dcc70 con 0x55a86169a2f0 2024-05-14T17:03:27.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.480+0000 7f2b4cff9700 10 client.15114 handle_command_reply: tid=0 2024-05-14T17:03:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.480+0000 7f2b760dab80 1 client.15114 shutdown 2024-05-14T17:03:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 msgr2=0x55a86168ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 0x55a86168ca70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b540 tx=0x7f2b54004750 comp rx=0 tx=0).stop 2024-05-14T17:03:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 shutdown_connections 2024-05-14T17:03:27.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a86168a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86169a2f0 0x55a86169b760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8616811d0 0x55a86168ca70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3253100134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686380 0x55a861689e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 >> 172.21.15.99:0/3253100134 conn(0x55a8615a9d60 msgr2=0x55a861676630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:27.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b4cff9700 0 client.15114 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:03:27.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 shutdown_connections 2024-05-14T17:03:27.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 1 -- 172.21.15.99:0/3253100134 wait complete. 2024-05-14T17:03:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.481+0000 7f2b760dab80 20 client.15114 trim_cache size 0 max 16384 2024-05-14T17:03:27.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 Processor -- start 2024-05-14T17:03:27.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 -- start start 2024-05-14T17:03:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8616811d0 0x55a861684560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a861678b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686ca0 0x55a861676db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a861699aa0 con 0x55a861686380 2024-05-14T17:03:27.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a86169a170 con 0x55a861686ca0 2024-05-14T17:03:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b760dab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a86169a840 con 0x55a8616811d0 2024-05-14T17:03:27.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a861678b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:27.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a861678b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36614/0 (socket says 172.21.15.99:36614) 2024-05-14T17:03:27.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3327946676 learned_addr learned my addr 172.21.15.99:0/3327946676 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:27.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b695e0700 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8616811d0 0x55a861684560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3327946676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8616811d0 msgr2=0x55a861684560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:27.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8616811d0 0x55a861684560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3327946676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686ca0 msgr2=0x55a861676db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:27.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b69de1700 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686ca0 0x55a861676db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:27.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686ca0 0x55a861676db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 -- 172.21.15.99:0/3327946676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a861680e00 con 0x55a861686380 2024-05-14T17:03:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b695e0700 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8616811d0 0x55a861684560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:27.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.502+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a861678b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000bcf0 tx=0x7f2b600107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378bb3f48b7a26c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:27.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b4effd700 1 -- 172.21.15.99:0/3327946676 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6006c020 con 0x55a861686380 2024-05-14T17:03:27.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b4effd700 1 -- 172.21.15.99:0/3327946676 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600a5630 con 0x55a861686380 2024-05-14T17:03:27.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b4effd700 1 -- 172.21.15.99:0/3327946676 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600c2030 con 0x55a861686380 2024-05-14T17:03:27.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 -- 172.21.15.99:0/3327946676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 msgr2=0x55a861678b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:27.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a861678b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000bcf0 tx=0x7f2b600107c0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 -- 172.21.15.99:0/3327946676 shutdown_connections 2024-05-14T17:03:27.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a861686ca0 0x55a861676db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8616811d0 0x55a861684560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 --2- 172.21.15.99:0/3327946676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a861678b80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 -- 172.21.15.99:0/3327946676 >> 172.21.15.99:0/3327946676 conn(0x55a8615a9d60 msgr2=0x55a861683e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 -- 172.21.15.99:0/3327946676 shutdown_connections 2024-05-14T17:03:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.503+0000 7f2b760dab80 1 -- 172.21.15.99:0/3327946676 wait complete. 2024-05-14T17:03:27.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 -- start start 2024-05-14T17:03:27.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 Processor -- start 2024-05-14T17:03:27.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a8616763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a8616768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a861676e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a86169b5e0 con 0x55a861686380 2024-05-14T17:03:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a86169af10 con 0x55a8616811d0 2024-05-14T17:03:27.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a86169bcb0 con 0x55a861686ca0 2024-05-14T17:03:27.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b760dab80 10 client.0 _ll_register_callbacks cb 0x55a861565900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a8616768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:27.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b68ddf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a8616768e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36628/0 (socket says 172.21.15.99:36628) 2024-05-14T17:03:27.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b68ddf700 1 -- 172.21.15.99:0/2639750663 learned_addr learned my addr 172.21.15.99:0/2639750663 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.504+0000 7f2b695e0700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a8616763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:27.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b68ddf700 1 -- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 msgr2=0x55a861676e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:27.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a861676e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b68ddf700 1 -- 172.21.15.99:0/2639750663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 msgr2=0x55a8616763a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a8616763a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b68ddf700 1 -- 172.21.15.99:0/2639750663 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8616a3300 con 0x55a861686380 2024-05-14T17:03:27.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b695e0700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a8616763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b68ddf700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a8616768e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2b600a9200 tx=0x7f2b600b1f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c05f528a62d3740 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:27.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:03:27.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b4cff9700 1 -- 172.21.15.99:0/2639750663 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600c2070 con 0x55a861686380 2024-05-14T17:03:28.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b4cff9700 1 -- 172.21.15.99:0/2639750663 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f2b600b1c10 con 0x55a861686380 2024-05-14T17:03:28.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b600a5630 con 0x55a861686380 2024-05-14T17:03:28.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b4cff9700 1 -- 172.21.15.99:0/2639750663 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b6006c040 con 0x55a861686380 2024-05-14T17:03:28.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 10 client.15120 fetch_fsmap learned FSMap version 25 2024-05-14T17:03:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a861705b50 con 0x55a861686380 2024-05-14T17:03:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b4cff9700 1 -- 172.21.15.99:0/2639750663 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b600a5630 con 0x55a861686380 2024-05-14T17:03:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b4cff9700 1 -- 172.21.15.99:0/2639750663 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2b600cb040 con 0x55a861686380 2024-05-14T17:03:28.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 10 client.15120 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T17:03:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T17:03:28.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 20 client.15120 populate_metadata read hostname 'smithi099' 2024-05-14T17:03:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86170a1c0 0x55a861708ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:28.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 4 client.15120 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:03:28.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.505+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "status"}) -- 0x55a8615d6450 con 0x55a86170a1c0 2024-05-14T17:03:28.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b695e0700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86170a1c0 0x55a861708ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:28.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b4cff9700 10 client.15120 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:03:28.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b695e0700 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86170a1c0 0x55a861708ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd68f74c170d3a7 server_cookie=e43595bb631fb546 in_seq=0 out_seq=0 2024-05-14T17:03:28.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b4cff9700 1 -- 172.21.15.99:0/2639750663 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a8615d6450 con 0x55a86170a1c0 2024-05-14T17:03:28.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b4cff9700 10 client.15120 handle_command_reply: tid=0 2024-05-14T17:03:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 client.15120 shutdown 2024-05-14T17:03:28.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 msgr2=0x55a8616768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:28.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a8616768e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2b600a9200 tx=0x7f2b600b1f20 comp rx=0 tx=0).stop 2024-05-14T17:03:28.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 shutdown_connections 2024-05-14T17:03:28.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a861686ca0 0x55a861676e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8616811d0 0x55a8616763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a86170a1c0 0x55a861708ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 --2- 172.21.15.99:0/2639750663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a861686380 0x55a8616768e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 >> 172.21.15.99:0/2639750663 conn(0x55a8615a9d60 msgr2=0x55a861689bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.506+0000 7f2b4cff9700 0 client.15120 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:03:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 shutdown_connections 2024-05-14T17:03:28.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b760dab80 1 -- 172.21.15.99:0/2639750663 wait complete. 2024-05-14T17:03:28.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b760dab80 20 client.15120 trim_cache size 0 max 16384 2024-05-14T17:03:28.413 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:03:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:03:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-05-14T17:03:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24463, 2024-05-14T17:03:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:03:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:03:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:03:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 193.45295805800001, 2024-05-14T17:03:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-05-14T17:03:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 82, 2024-05-14T17:03:28.532 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T17:03:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 220.53548757199999 2024-05-14T17:03:28.551 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:03:28.603 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:26 smithi193 ceph-mon[35962]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:28.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:28 smithi193 ceph-mon[35962]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:28.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:26 smithi150 ceph-mon[36204]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:28.644 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:28 smithi150 ceph-mon[36204]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:28.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2b500b2880 msgr2=0x7f2b500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:28.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2b500b2880 0x7f2b500b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005950 tx=0x7f2b540058e0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 msgr2=0x7f2b6419b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:28.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6419b030 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000b530 tx=0x7f2b6000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 shutdown_connections 2024-05-14T17:03:28.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f2b500b2880 0x7f2b500b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b641073c0 0x7f2b641a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b64102470 0x7f2b6419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 --2- 172.21.15.99:0/1179037952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b64102e30 0x7f2b6419b030 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:28.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 >> 172.21.15.99:0/1179037952 conn(0x7f2b640784e0 msgr2=0x7f2b641005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:28.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 shutdown_connections 2024-05-14T17:03:28.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:25.507+0000 7f2b6b844700 1 -- 172.21.15.99:0/1179037952 wait complete. 2024-05-14T17:03:28.825 INFO:teuthology.run_tasks:Running task sequential... 2024-05-14T17:03:28.897 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-14T17:03:28.911 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-14T17:03:28.924 INFO:teuthology.task.print:Enabling mgr modules 2024-05-14T17:03:28.935 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-14T17:03:28.948 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T17:03:28.963 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:03:28.974 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-14T17:03:28.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:26 smithi099 ceph-mon[31851]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T17:03:28.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:28 smithi099 ceph-mon[31851]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:03:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 -- 172.21.15.99:0/3931522311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db81088a0 msgr2=0x7f5db810ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:29.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 --2- 172.21.15.99:0/3931522311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db81088a0 0x7f5db810ac90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009900 tx=0x7f5db002a410 comp rx=0 tx=0).stop 2024-05-14T17:03:29.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 -- 172.21.15.99:0/3931522311 shutdown_connections 2024-05-14T17:03:29.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 --2- 172.21.15.99:0/3931522311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db81088a0 0x7f5db810ac90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 --2- 172.21.15.99:0/3931522311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5db8105f70 0x7f5db8108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 --2- 172.21.15.99:0/3931522311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5db8103640 0x7f5db8105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 -- 172.21.15.99:0/3931522311 >> 172.21.15.99:0/3931522311 conn(0x7f5db80fcfd0 msgr2=0x7f5db80ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 -- 172.21.15.99:0/3931522311 shutdown_connections 2024-05-14T17:03:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 -- 172.21.15.99:0/3931522311 wait complete. 2024-05-14T17:03:29.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.130+0000 7f5dbf620640 1 Processor -- start 2024-05-14T17:03:29.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 -- start start 2024-05-14T17:03:29.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 0x7f5db8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:29.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5db8105f70 0x7f5db81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:29.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5db81088a0 0x7f5db81a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:29.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810e6e0 con 0x7f5db8103640 2024-05-14T17:03:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810e560 con 0x7f5db81088a0 2024-05-14T17:03:29.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810e860 con 0x7f5db8105f70 2024-05-14T17:03:29.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 0x7f5db8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:29.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 0x7f5db8198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36640/0 (socket says 172.21.15.99:36640) 2024-05-14T17:03:29.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 -- 172.21.15.99:0/543959033 learned_addr learned my addr 172.21.15.99:0/543959033 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 -- 172.21.15.99:0/543959033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5db8105f70 msgr2=0x7f5db81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:29.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5db8105f70 0x7f5db81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 -- 172.21.15.99:0/543959033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5db81088a0 msgr2=0x7f5db81a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:29.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5db81088a0 0x7f5db81a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:29.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 -- 172.21.15.99:0/543959033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db81a0870 con 0x7f5db8103640 2024-05-14T17:03:29.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbd3bc640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 0x7f5db8198c60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5db400cce0 tx=0x7f5db400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:29.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4004b20 con 0x7f5db8103640 2024-05-14T17:03:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db4004cc0 con 0x7f5db8103640 2024-05-14T17:03:29.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db81a0b60 con 0x7f5db8103640 2024-05-14T17:03:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db400f6b0 con 0x7f5db8103640 2024-05-14T17:03:29.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.131+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81a1180 con 0x7f5db8103640 2024-05-14T17:03:29.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.132+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db80feac0 con 0x7f5db8103640 2024-05-14T17:03:29.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.134+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5db400f850 con 0x7f5db8103640 2024-05-14T17:03:29.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.135+0000 7f5dae7fc640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5da40b2920 0x7f5da40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:29.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.135+0000 7f5dbcbbb640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5da40b2920 0x7f5da40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.135+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f5db40e2750 con 0x7f5db8103640 2024-05-14T17:03:29.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.136+0000 7f5dbcbbb640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5da40b2920 0x7f5da40b4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5db806bd80 tx=0x7f5da8006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:29.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.139+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5db4014030 con 0x7f5db8103640 2024-05-14T17:03:29.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:29.373+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f5db802de30 con 0x7f5db8103640 2024-05-14T17:03:30.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.163+0000 7f5dae7fc640 1 -- 172.21.15.99:0/543959033 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f5db40aa500 con 0x7f5db8103640 2024-05-14T17:03:30.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5da40b2920 msgr2=0x7f5da40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5da40b2920 0x7f5da40b4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5db806bd80 tx=0x7f5da8006c90 comp rx=0 tx=0).stop 2024-05-14T17:03:30.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 msgr2=0x7f5db8198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 0x7f5db8198c60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5db400cce0 tx=0x7f5db400a620 comp rx=0 tx=0).stop 2024-05-14T17:03:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 shutdown_connections 2024-05-14T17:03:30.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5db81088a0 0x7f5db81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:30.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5db8105f70 0x7f5db81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:30.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f5da40b2920 0x7f5da40b4de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:30.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 --2- 172.21.15.99:0/543959033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db8103640 0x7f5db8198c60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:30.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 >> 172.21.15.99:0/543959033 conn(0x7f5db80fcfd0 msgr2=0x7f5db8103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:30.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 shutdown_connections 2024-05-14T17:03:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:30.166+0000 7f5dbf620640 1 -- 172.21.15.99:0/543959033 wait complete. 2024-05-14T17:03:30.514 INFO:teuthology.run_tasks:Running task kclient... 2024-05-14T17:03:30.618 INFO:tasks.kclient:Mounting kernel clients... 2024-05-14T17:03:30.627 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-14T17:03:30.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:30.682 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-05-14T17:03:30.720 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:30 smithi193 ceph-mon[35962]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:30.773 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:30 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/543959033' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T17:03:30.900 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:30 smithi150 ceph-mon[36204]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:30.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:30 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/543959033' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T17:03:30.986 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:30 smithi099 ceph-mon[31851]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T17:03:30.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/543959033' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T17:03:31.011 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: ignoring --setuser ceph since I am not root 2024-05-14T17:03:31.020 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: ignoring --setgroup ceph since I am not root 2024-05-14T17:03:31.033 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-mgr[37812]: -- 172.21.15.193:0/1823338054 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d13bbec5a0 con 0x55d13bbc0c00 2024-05-14T17:03:31.043 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:30.350+0000 7f307d944200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T17:03:31.057 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:30.534+0000 7f307d944200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T17:03:31.071 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:30.636+0000 7f307d944200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T17:03:31.083 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: ignoring --setuser ceph since I am not root 2024-05-14T17:03:31.096 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: ignoring --setgroup ceph since I am not root 2024-05-14T17:03:31.107 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-mgr[37588]: -- 172.21.15.150:0/2759924771 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55691283e5a0 con 0x556912812c00 2024-05-14T17:03:31.117 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:30.346+0000 7fc2b6d11200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T17:03:31.131 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:30.532+0000 7fc2b6d11200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T17:03:31.145 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:30.635+0000 7fc2b6d11200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T17:03:31.194 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:30.945+0000 7fc2b6d11200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T17:03:31.206 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: ignoring --setuser ceph since I am not root 2024-05-14T17:03:31.218 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: ignoring --setgroup ceph since I am not root 2024-05-14T17:03:31.307 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:30.347+0000 7f8f1863c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T17:03:31.320 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:30.533+0000 7f8f1863c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T17:03:31.332 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:30.639+0000 7f8f1863c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T17:03:31.343 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:30.944+0000 7f307d944200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T17:03:31.356 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:31.028+0000 7f307d944200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T17:03:31.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:31 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/543959033' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T17:03:31.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:31 smithi150 ceph-mon[36204]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T17:03:31.395 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:31 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/543959033' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T17:03:31.409 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:31 smithi193 ceph-mon[35962]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T17:03:31.422 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:31.031+0000 7fc2b6d11200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T17:03:31.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:31.446 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-05-14T17:03:31.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:31 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/543959033' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T17:03:31.469 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:31 smithi099 ceph-mon[31851]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T17:03:31.505 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:30.957+0000 7f8f1863c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T17:03:31.513 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:31.045+0000 7f8f1863c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T17:03:31.549 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-05-14T17:03:31.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:03:31.592 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-14T17:03:31.604 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T17:03:31.615 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T17:03:31.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T17:03:31.891 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:32.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:32.210+0000 7f307d944200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T17:03:32.555 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:32.381+0000 7f307d944200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T17:03:32.563 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:32.220+0000 7fc2b6d11200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T17:03:32.570 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:32.390+0000 7fc2b6d11200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T17:03:32.619 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:32.561+0000 7fc2b6d11200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T17:03:32.644 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:32.292+0000 7f8f1863c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T17:03:32.650 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:32.467+0000 7f8f1863c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T17:03:32.656 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:32.644+0000 7f8f1863c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T17:03:32.844 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:32.553+0000 7f307d944200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T17:03:33.141 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:32.844+0000 7f307d944200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T17:03:33.150 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:32.937+0000 7f307d944200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T17:03:33.158 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:32.850+0000 7fc2b6d11200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T17:03:33.165 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:32.942+0000 7fc2b6d11200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T17:03:33.173 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:33.145+0000 7fc2b6d11200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T17:03:33.231 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:32.941+0000 7f8f1863c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T17:03:33.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:33.043+0000 7f8f1863c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T17:03:33.453 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:33.338+0000 7fc2b6d11200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T17:03:33.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 -- 172.21.15.99:0/614808756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050108520 msgr2=0x7f8050108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:33.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 --2- 172.21.15.99:0/614808756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050108520 0x7f8050108900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8038009990 tx=0x7f8038009ca0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 -- 172.21.15.99:0/614808756 shutdown_connections 2024-05-14T17:03:33.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 --2- 172.21.15.99:0/614808756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102e00 0x7f805010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 --2- 172.21.15.99:0/614808756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050108520 0x7f8050108900 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 --2- 172.21.15.99:0/614808756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050102440 0x7f80501028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 -- 172.21.15.99:0/614808756 >> 172.21.15.99:0/614808756 conn(0x7f80500793f0 msgr2=0x7f8050079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 -- 172.21.15.99:0/614808756 shutdown_connections 2024-05-14T17:03:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 -- 172.21.15.99:0/614808756 wait complete. 2024-05-14T17:03:33.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 Processor -- start 2024-05-14T17:03:33.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.477+0000 7f80564b4700 1 -- start start 2024-05-14T17:03:33.651 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:33.141+0000 7f307d944200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T17:03:33.659 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:33.338+0000 7f307d944200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T17:03:33.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f80564b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102440 0x7f8050104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f80564b4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 0x7f80501045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f80564b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050108520 0x7f8050104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f80564b4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80501125f0 con 0x7f8050102e00 2024-05-14T17:03:33.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f80564b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8050112470 con 0x7f8050108520 2024-05-14T17:03:33.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f80564b4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8050112770 con 0x7f8050102440 2024-05-14T17:03:33.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 0x7f80501045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 0x7f80501045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36674/0 (socket says 172.21.15.99:36674) 2024-05-14T17:03:33.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 -- 172.21.15.99:0/2931872813 learned_addr learned my addr 172.21.15.99:0/2931872813 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:33.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804ffff700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102440 0x7f8050104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f8054a51700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050108520 0x7f8050104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 -- 172.21.15.99:0/2931872813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102440 msgr2=0x7f8050104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:33.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102440 0x7f8050104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 -- 172.21.15.99:0/2931872813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050108520 msgr2=0x7f8050104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:33.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050108520 0x7f8050104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804f7fe700 1 -- 172.21.15.99:0/2931872813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80501a8120 con 0x7f8050102e00 2024-05-14T17:03:33.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.478+0000 7f804ffff700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102440 0x7f8050104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.479+0000 7f804f7fe700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 0x7f80501045c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f804000e9a0 tx=0x7f804000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:33.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.479+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000ca10 con 0x7f8050102e00 2024-05-14T17:03:33.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.479+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f804000cbb0 con 0x7f8050102e00 2024-05-14T17:03:33.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.479+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040018680 con 0x7f8050102e00 2024-05-14T17:03:33.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.479+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80501a8410 con 0x7f8050102e00 2024-05-14T17:03:33.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.479+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80501a8900 con 0x7f8050102e00 2024-05-14T17:03:33.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.480+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805006a230 con 0x7f8050102e00 2024-05-14T17:03:33.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.482+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f8040010430 con 0x7f8050102e00 2024-05-14T17:03:33.975 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:33.254+0000 7f8f1863c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T17:03:33.982 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:33.458+0000 7f8f1863c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T17:03:33.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.483+0000 7f804d7fa700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 0x7f803c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:34.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.483+0000 7f804ffff700 1 -- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 msgr2=0x7f803c0b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2498052825 2024-05-14T17:03:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.483+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8219+0+0 (secure 0 0 0) 0x7f80400e2710 con 0x7f8050102e00 2024-05-14T17:03:34.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.483+0000 7f804ffff700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 0x7f803c0b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T17:03:34.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.485+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80400aa440 con 0x7f8050102e00 2024-05-14T17:03:34.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.684+0000 7f804ffff700 1 -- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 msgr2=0x7f803c0b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2498052825 2024-05-14T17:03:34.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.684+0000 7f804ffff700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 0x7f803c0b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T17:03:34.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.743+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80501a8b60 con 0x7f8050102e00 2024-05-14T17:03:34.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.743+0000 7f804d7fa700 1 -- 172.21.15.99:0/2931872813 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f80400a9d40 con 0x7f8050102e00 2024-05-14T17:03:34.104 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T17:03:34.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 msgr2=0x7f803c0b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:03:34.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 0x7f803c0b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 msgr2=0x7f80501045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:34.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 0x7f80501045c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f804000e9a0 tx=0x7f804000ecb0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 shutdown_connections 2024-05-14T17:03:34.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:6800/2498052825,v1:172.21.15.99:6801/2498052825] conn(0x7f803c0b28d0 0x7f803c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050108520 0x7f8050104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8050102440 0x7f8050104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 --2- 172.21.15.99:0/2931872813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8050102e00 0x7f80501045c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 >> 172.21.15.99:0/2931872813 conn(0x7f80500793f0 msgr2=0x7f805006b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:34.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 shutdown_connections 2024-05-14T17:03:34.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:33.746+0000 7f80564b4700 1 -- 172.21.15.99:0/2931872813 wait complete. 2024-05-14T17:03:34.307 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:33 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2931872813' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T17:03:34.314 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:33 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2931872813' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T17:03:34.321 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:33.940+0000 7f307d944200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T17:03:34.327 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:33.946+0000 7fc2b6d11200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T17:03:34.334 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:34.168+0000 7fc2b6d11200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T17:03:34.341 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:33 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2931872813' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T17:03:34.362 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:34.093+0000 7f8f1863c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T17:03:34.367 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:34.322+0000 7f8f1863c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T17:03:34.375 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T17:03:34.381 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T17:03:34.388 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-05-14T17:03:34.394 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T17:03:34.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T17:03:34.406 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T17:03:34.438 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T17:03:34.498 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T17:03:34.508 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T17:03:34.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:34.523 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-05-14T17:03:34.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:34.160+0000 7f307d944200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T17:03:34.548 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T17:03:34.555 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T17:03:34.561 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T17:03:34.578 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-05-14T17:03:34.584 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-05-14T17:03:34.592 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-05-14T17:03:34.604 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T17:03:34.612 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-05-14T17:03:34.620 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-05-14T17:03:34.627 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T17:03:34.636 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-05-14T17:03:34.673 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-05-14T17:03:34.706 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T17:03:34.716 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-05-14T17:03:34.725 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T17:03:34.732 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-05-14T17:03:34.741 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-14T17:03:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 41979sec preferred_lft 41979sec 2024-05-14T17:03:34.759 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-05-14T17:03:34.767 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-05-14T17:03:34.779 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T17:03:34.789 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T17:03:34.795 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-05-14T17:03:34.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-05-14T17:03:34.816 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-05-14T17:03:34.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-05-14T17:03:34.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T17:03:34.844 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-05-14T17:03:34.953 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:34.594+0000 7fc2b6d11200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T17:03:34.959 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:34.691+0000 7fc2b6d11200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T17:03:34.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T17:03:35.042 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:34.588+0000 7f307d944200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T17:03:35.048 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:34.686+0000 7f307d944200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T17:03:35.058 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:34.764+0000 7f8f1863c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T17:03:35.065 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:34.864+0000 7f8f1863c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T17:03:35.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:35.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:35.123 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T17:03:35.156 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-05-14T17:03:35.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:35.176 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-05-14T17:03:35.244 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-05-14T17:03:35.252 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T17:03:35.262 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-14T17:03:35.268 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-14T17:03:35.339 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T17:03:35.346 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T17:03:35.354 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-05-14T17:03:35.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-14T17:03:35.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-14T17:03:35.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-14T17:03:35.382 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-05-14T17:03:35.423 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:35.062+0000 7f307d944200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T17:03:35.431 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:35.067+0000 7fc2b6d11200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T17:03:35.446 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:35.426+0000 7fc2b6d11200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T17:03:35.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T17:03:35.462 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:35.258+0000 7f8f1863c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T17:03:35.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:35.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:35.651 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-05-14T17:03:35.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:35.681 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-05-14T17:03:35.751 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:35.520+0000 7fc2b6d11200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T17:03:35.758 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:35.636+0000 7f8f1863c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T17:03:35.765 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:35.422+0000 7f307d944200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T17:03:35.771 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:35.517+0000 7f307d944200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T17:03:35.780 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T17:03:35.787 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-05-14T17:03:35.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T17:03:35.801 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-14T17:03:35.807 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-05-14T17:03:35.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T17:03:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:35.913 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-14T17:03:35.922 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T17:03:35.930 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-05-14T17:03:35.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-14T17:03:35.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T17:03:36.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-14T17:03:36.089 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-14T17:03:36.102 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-14T17:03:36.111 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-05-14T17:03:36.121 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:35.761+0000 7f307d944200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T17:03:36.130 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:35.895+0000 7f307d944200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T17:03:36.143 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:35.989+0000 7f307d944200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T17:03:36.150 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:35.735+0000 7f8f1863c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T17:03:36.203 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:35.765+0000 7fc2b6d11200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T17:03:36.211 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:35.900+0000 7fc2b6d11200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T17:03:36.247 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:35.997+0000 7fc2b6d11200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T17:03:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T17:03:36.289 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:35.996+0000 7f8f1863c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T17:03:36.296 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:36.135+0000 7f8f1863c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T17:03:36.304 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:36.239+0000 7f8f1863c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T17:03:36.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:36.320 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T17:03:36.327 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-05-14T17:03:36.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-05-14T17:03:36.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T17:03:36.381 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-05-14T17:03:36.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T17:03:36.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:36.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T17:03:36.538 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T17:03:36.562 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T17:03:36.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T17:03:36.590 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T17:03:36.617 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-14T17:03:36.627 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-14T17:03:36.634 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/74f2e792-4109-459b-a6e4-7da494b4c0cd 2024-05-14T17:03:36.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:36.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/74f2e792-4109-459b-a6e4-7da494b4c0cd /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-14T17:03:36.703 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:36.328+0000 7fc2b6d11200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T17:03:36.710 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:36.421+0000 7fc2b6d11200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T17:03:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-14T17:03:36.771 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-14T17:03:36.792 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:36.321+0000 7f307d944200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T17:03:36.801 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:36.414+0000 7f307d944200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T17:03:36.834 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:36.583+0000 7f8f1863c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T17:03:36.917 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:36.686+0000 7f8f1863c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T17:03:36.927 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-14T17:03:36.937 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-14T17:03:36.947 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.150:3300,172.21.15.193:3300" 2024-05-14T17:03:36.966 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@2a006d80-1212-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/74f2e792-4109-459b-a6e4-7da494b4c0cd 2024-05-14T17:03:36.976 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.99:3300/172.21.15.150:3300/172.21.15.193:3300" will pass to kernel 2024-05-14T17:03:36.985 INFO:teuthology.orchestra.run.smithi099.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-14T17:03:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: You just mounted a file system that supports labels which does not 2024-05-14T17:03:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-14T17:03:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: applications will generate AVC messages and not be allowed access to 2024-05-14T17:03:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-14T17:03:37.030 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-14T17:03:37.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:03:37.119 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T17:03:37.156 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T17:03:37.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T17:03:37.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:37.318 INFO:teuthology.orchestra.run.smithi099.stdout:client24823 2024-05-14T17:03:37.328 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T17:03:37.337 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:03:37.346 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:37.024+0000 7fc2b6d11200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T17:03:37.379 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:03:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y[37584]: 2024-05-14T17:03:37.242+0000 7fc2b6d11200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T17:03:37.387 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:37.020+0000 7f307d944200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T17:03:37.396 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:03:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z[37808]: 2024-05-14T17:03:37.241+0000 7f307d944200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T17:03:37.421 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T17:03:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T17:03:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:37.528 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.331+0000 7f8f1863c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T17:03:37.538 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: Standby manager daemon y restarted 2024-05-14T17:03:37.602 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: Standby manager daemon y started 2024-05-14T17:03:37.682 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: Standby manager daemon z restarted 2024-05-14T17:03:37.690 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: Standby manager daemon z started 2024-05-14T17:03:37.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T17:03:37.703 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T17:03:37.709 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T17:03:37.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T17:03:37.725 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T17:03:37.732 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T17:03:37.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T17:03:37.763 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:37 smithi099 ceph-mon[31851]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T17:03:37.771 INFO:teuthology.orchestra.run.smithi099.stdout:client24823 2024-05-14T17:03:37.778 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T17:03:37.785 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: Standby manager daemon y restarted 2024-05-14T17:03:37.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: Standby manager daemon y started 2024-05-14T17:03:37.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: Standby manager daemon z restarted 2024-05-14T17:03:37.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: Standby manager daemon z started 2024-05-14T17:03:37.852 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T17:03:37.859 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T17:03:37.958 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T17:03:37.966 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T17:03:37.973 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T17:03:37.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T17:03:37.986 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T17:03:37.993 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:37 smithi193 ceph-mon[35962]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T17:03:38.001 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: Standby manager daemon y restarted 2024-05-14T17:03:38.041 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: Standby manager daemon y started 2024-05-14T17:03:38.050 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: Standby manager daemon z restarted 2024-05-14T17:03:38.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: Standby manager daemon z started 2024-05-14T17:03:38.094 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T17:03:38.105 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T17:03:38.178 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T17:03:38.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T17:03:38.193 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T17:03:38.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.150:0/530501017' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T17:03:38.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T17:03:38.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:37 smithi150 ceph-mon[36204]: from='mgr.? 172.21.15.193:0/2592188152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T17:03:38.258 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.564+0000 7f8f1863c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T17:03:38.267 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e78653700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.274 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e78653700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.281 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e78653700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.288 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e78653700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.294 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e78653700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.316 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e75e4e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.324 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e75e4e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.331 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e75e4e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.341 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e75e4e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.348 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:03:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:03:37.824+0000 7f8e75e4e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:03:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T17:03:38.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:38.453 INFO:teuthology.orchestra.run.smithi099.stdout:client24823 2024-05-14T17:03:38.486 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2a006d80-1212-11ef-bc98-c7b262605968.client24823/status') 2024-05-14T17:03:38.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T17:03:38.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-14T17:03:38.567 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: Active manager daemon x restarted 2024-05-14T17:03:38.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: Activating manager daemon x 2024-05-14T17:03:38.584 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T17:03:38.595 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: mgrmap e21: x(active, starting, since 0.0115264s), standbys: y, z 2024-05-14T17:03:38.605 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T17:03:38.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T17:03:38.632 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T17:03:38.639 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T17:03:38.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T17:03:38.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T17:03:38.675 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T17:03:38.728 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T17:03:38.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:03:38.748 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:03:38.775 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:03:38.783 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:03:38.790 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:03:38.799 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:03:38.807 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:03:38.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T17:03:38.825 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T17:03:38.834 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T17:03:38.842 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T17:03:38.858 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T17:03:38.866 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T17:03:38.875 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T17:03:38.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T17:03:38.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T17:03:38.953 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T17:03:38.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T17:03:38.974 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T17:03:39.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T17:03:39.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T17:03:39.017 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T17:03:39.026 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T17:03:39.032 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T17:03:39.038 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T17:03:39.047 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: Manager daemon x is now available 2024-05-14T17:03:39.055 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:39.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T17:03:39.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T17:03:39.083 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: [14/May/2024:17:03:38] ENGINE Bus STARTING 2024-05-14T17:03:39.091 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:38 smithi099 ceph-mon[31851]: [14/May/2024:17:03:38] ENGINE Serving on https://172.21.15.99:7150 2024-05-14T17:03:39.100 INFO:teuthology.orchestra.run.smithi099.stdout:instance: client.24823 (3)172.21.15.99:0/3159274093 2024-05-14T17:03:39.108 INFO:teuthology.orchestra.run.smithi099.stdout:blocklisted: false 2024-05-14T17:03:39.198 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-14T17:03:39.212 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: Active manager daemon x restarted 2024-05-14T17:03:39.220 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: Activating manager daemon x 2024-05-14T17:03:39.261 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T17:03:39.269 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: mgrmap e21: x(active, starting, since 0.0115264s), standbys: y, z 2024-05-14T17:03:39.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T17:03:39.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T17:03:39.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T17:03:39.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T17:03:39.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T17:03:39.321 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T17:03:39.329 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T17:03:39.342 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T17:03:39.349 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:03:39.355 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:03:39.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:03:39.370 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:03:39.450 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:03:39.457 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:03:39.465 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:03:39.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T17:03:39.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T17:03:39.523 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T17:03:39.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T17:03:39.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T17:03:39.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T17:03:39.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T17:03:39.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T17:03:39.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T17:03:39.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T17:03:39.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T17:03:39.608 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T17:03:39.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T17:03:39.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T17:03:39.696 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T17:03:39.703 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T17:03:39.711 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T17:03:39.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T17:03:39.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: Manager daemon x is now available 2024-05-14T17:03:39.754 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:39.763 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T17:03:39.772 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T17:03:39.784 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: [14/May/2024:17:03:38] ENGINE Bus STARTING 2024-05-14T17:03:39.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:38 smithi193 ceph-mon[35962]: [14/May/2024:17:03:38] ENGINE Serving on https://172.21.15.99:7150 2024-05-14T17:03:39.800 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-14T17:03:39.809 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: Active manager daemon x restarted 2024-05-14T17:03:39.839 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: Activating manager daemon x 2024-05-14T17:03:39.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T17:03:39.858 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: mgrmap e21: x(active, starting, since 0.0115264s), standbys: y, z 2024-05-14T17:03:39.867 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T17:03:39.875 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T17:03:39.950 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T17:03:39.959 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T17:03:39.966 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T17:03:39.998 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T17:03:40.005 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T17:03:40.011 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T17:03:40.019 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:03:40.028 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:03:40.036 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:03:40.045 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:03:40.053 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:03:40.061 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:03:40.076 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:03:40.085 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T17:03:40.093 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T17:03:40.100 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T17:03:40.107 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T17:03:40.115 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T17:03:40.175 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T17:03:40.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T17:03:40.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T17:03:40.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T17:03:40.227 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T17:03:40.234 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T17:03:40.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T17:03:40.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T17:03:40.254 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T17:03:40.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T17:03:40.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T17:03:40.285 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T17:03:40.291 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T17:03:40.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: Manager daemon x is now available 2024-05-14T17:03:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:40.325 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T17:03:40.403 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T17:03:40.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: [14/May/2024:17:03:38] ENGINE Bus STARTING 2024-05-14T17:03:40.421 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:38 smithi150 ceph-mon[36204]: [14/May/2024:17:03:38] ENGINE Serving on https://172.21.15.99:7150 2024-05-14T17:03:40.455 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records in 2024-05-14T17:03:40.462 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records out 2024-05-14T17:03:40.468 INFO:teuthology.orchestra.run.smithi099.stderr:71 bytes copied, 2.2184e-05 s, 3.2 MB/s 2024-05-14T17:03:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T17:03:40.483 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T17:03:40.499 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T17:03:40.505 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:03:40.513 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-14T17:03:40.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: [14/May/2024:17:03:38] ENGINE Serving on http://172.21.15.99:8765 2024-05-14T17:03:40.545 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: [14/May/2024:17:03:38] ENGINE Bus STARTED 2024-05-14T17:03:40.552 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:40.559 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:40.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:40.574 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:40.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: mgrmap e22: x(active, since 1.02214s), standbys: y, z 2024-05-14T17:03:41.327 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:39 smithi099 ceph-mon[31851]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:41.418 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:41.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.442 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.451 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.470 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.478 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:40 smithi099 ceph-mon[31851]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-14T17:03:41.497 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: [14/May/2024:17:03:38] ENGINE Serving on http://172.21.15.99:8765 2024-05-14T17:03:41.545 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: [14/May/2024:17:03:38] ENGINE Bus STARTED 2024-05-14T17:03:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.645 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: mgrmap e22: x(active, since 1.02214s), standbys: y, z 2024-05-14T17:03:41.656 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:39 smithi193 ceph-mon[35962]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:41.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:41.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.725 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.752 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.763 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:40 smithi193 ceph-mon[35962]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-14T17:03:41.777 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: [14/May/2024:17:03:38] ENGINE Serving on http://172.21.15.99:8765 2024-05-14T17:03:41.792 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: [14/May/2024:17:03:38] ENGINE Bus STARTED 2024-05-14T17:03:41.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.831 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.862 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:41.874 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: mgrmap e22: x(active, since 1.02214s), standbys: y, z 2024-05-14T17:03:41.954 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:39 smithi150 ceph-mon[36204]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:41.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:41.977 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.042 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.065 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.078 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.089 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.103 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:40 smithi150 ceph-mon[36204]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-14T17:03:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 -- 172.21.15.99:0/3374684425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 msgr2=0x7f2ff010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:42.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 --2- 172.21.15.99:0/3374684425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff010ac20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009ca0 tx=0x7f2fe802a8c0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 -- 172.21.15.99:0/3374684425 shutdown_connections 2024-05-14T17:03:42.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 --2- 172.21.15.99:0/3374684425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff010ac20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 --2- 172.21.15.99:0/3374684425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff0105f00 0x7f2ff01082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 --2- 172.21.15.99:0/3374684425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff01035d0 0x7f2ff01059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 -- 172.21.15.99:0/3374684425 >> 172.21.15.99:0/3374684425 conn(0x7f2ff00fcf60 msgr2=0x7f2ff00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 -- 172.21.15.99:0/3374684425 shutdown_connections 2024-05-14T17:03:42.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.524+0000 7f2ff710a640 1 -- 172.21.15.99:0/3374684425 wait complete. 2024-05-14T17:03:42.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 Processor -- start 2024-05-14T17:03:42.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 -- start start 2024-05-14T17:03:42.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff01035d0 0x7f2ff0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:42.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff0105f00 0x7f2ff0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:42.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff010e620 con 0x7f2ff0108830 2024-05-14T17:03:42.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff010e4a0 con 0x7f2ff01035d0 2024-05-14T17:03:42.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff710a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff010e7a0 con 0x7f2ff0105f00 2024-05-14T17:03:42.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff56a7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff4ea6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff01035d0 0x7f2ff0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff56a7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff01a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54530/0 (socket says 172.21.15.99:54530) 2024-05-14T17:03:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2ff56a7640 1 -- 172.21.15.99:0/4133059080 learned_addr learned my addr 172.21.15.99:0/4133059080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.525+0000 7f2feffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff0105f00 0x7f2ff0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff4ea6640 1 -- 172.21.15.99:0/4133059080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff0105f00 msgr2=0x7f2ff0199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:42.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff4ea6640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff0105f00 0x7f2ff0199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff4ea6640 1 -- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 msgr2=0x7f2ff01a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:42.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff4ea6640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff01a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:42.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff4ea6640 1 -- 172.21.15.99:0/4133059080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01a0830 con 0x7f2ff01035d0 2024-05-14T17:03:42.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff56a7640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:42.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff4ea6640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff01035d0 0x7f2ff0198c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e9f0 tx=0x7f2fe400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:42.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400ccc0 con 0x7f2ff01035d0 2024-05-14T17:03:42.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff01a0b20 con 0x7f2ff01035d0 2024-05-14T17:03:42.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe40108e0 con 0x7f2ff01035d0 2024-05-14T17:03:42.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4018880 con 0x7f2ff01035d0 2024-05-14T17:03:42.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.526+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01a10c0 con 0x7f2ff01035d0 2024-05-14T17:03:42.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.527+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff00fea50 con 0x7f2ff01035d0 2024-05-14T17:03:42.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.529+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f2fe4010430 con 0x7f2ff01035d0 2024-05-14T17:03:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.529+0000 7f2fedffb640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2fe00b2a10 0x7f2fe00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:42.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.529+0000 7f2feffff640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2fe00b2a10 0x7f2fe00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:42.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.529+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2fe40e29b0 con 0x7f2ff01035d0 2024-05-14T17:03:42.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.530+0000 7f2feffff640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2fe00b2a10 0x7f2fe00b4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ff006bd60 tx=0x7f2fdc005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:42.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.533+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fe40a9d70 con 0x7f2ff01035d0 2024-05-14T17:03:42.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:41.793+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f2ff002dba0 con 0x7f2ff01035d0 2024-05-14T17:03:42.724 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.734 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.746 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.775 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.785 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi099 to 176.1M 2024-05-14T17:03:42.810 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Unable to set osd_memory_target on smithi099 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-14T17:03:42.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.829 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.840 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.851 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.863 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.873 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.884 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-14T17:03:42.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-14T17:03:42.906 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:42.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:42.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.001 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.009 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.018 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T17:03:43.027 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T17:03:43.037 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:43.047 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:43.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T17:03:43.065 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T17:03:43.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T17:03:43.097 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4133059080' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T17:03:43.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:41 smithi193 ceph-mon[35962]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T17:03:43.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.130 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.143 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.196 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi099 to 176.1M 2024-05-14T17:03:43.237 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Unable to set osd_memory_target on smithi099 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-14T17:03:43.252 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.288 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.300 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.314 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.324 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-14T17:03:43.363 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-14T17:03:43.376 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.389 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.404 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.419 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T17:03:43.485 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T17:03:43.497 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:43.510 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:43.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T17:03:43.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T17:03:43.542 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T17:03:43.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4133059080' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T17:03:43.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:41 smithi150 ceph-mon[36204]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T17:03:43.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.201+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f2fe40a5f70 con 0x7f2ff01035d0 2024-05-14T17:03:43.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.201+0000 7f2fedffb640 1 -- 172.21.15.99:0/4133059080 <== mon.2 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f2fe40db9b0 con 0x7f2ff01035d0 2024-05-14T17:03:43.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2fe00b2a10 msgr2=0x7f2fe00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:43.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2fe00b2a10 0x7f2fe00b4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ff006bd60 tx=0x7f2fdc005f90 comp rx=0 tx=0).stop 2024-05-14T17:03:43.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff01035d0 msgr2=0x7f2ff0198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:43.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff01035d0 0x7f2ff0198c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e9f0 tx=0x7f2fe400eeb0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 shutdown_connections 2024-05-14T17:03:43.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2fe00b2a10 0x7f2fe00b4ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff0108830 0x7f2ff01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff0105f00 0x7f2ff0199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 --2- 172.21.15.99:0/4133059080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff01035d0 0x7f2ff0198c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:43.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 >> 172.21.15.99:0/4133059080 conn(0x7f2ff00fcf60 msgr2=0x7f2ff0103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:43.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 shutdown_connections 2024-05-14T17:03:43.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:42.205+0000 7f2ff710a640 1 -- 172.21.15.99:0/4133059080 wait complete. 2024-05-14T17:03:43.834 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.845 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.856 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.865 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.889 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.900 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:43.912 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi099 to 176.1M 2024-05-14T17:03:43.922 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Unable to set osd_memory_target on smithi099 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-14T17:03:43.930 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.982 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:43.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.040 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.048 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.058 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-14T17:03:44.070 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-14T17:03:44.080 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.091 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.099 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:03:44.170 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T17:03:44.180 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T17:03:44.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:44.210 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:44.271 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-14T17:03:44.278 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-14T17:03:44.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T17:03:44.313 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4133059080' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T17:03:44.327 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:41 smithi099 ceph-mon[31851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T17:03:44.339 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-14T17:03:44.364 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T17:03:44.471 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T17:03:44.491 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:03:44.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-14T17:03:44.511 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:44.524 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:44.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:44.578 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:44.591 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:44.601 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:44.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:44.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:44.658 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:44.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:44.743 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T17:03:44.754 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:03:44.766 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-14T17:03:44.778 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.791 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.803 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.825 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.839 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.853 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.865 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:44.878 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:44.892 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:44.902 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:43 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:44.961 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:44.968 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:44.975 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:45.003 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:45.011 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:45.023 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:45.033 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:45.044 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:45.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:45.067 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:45.077 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T17:03:45.087 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:03:45.118 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-14T17:03:45.128 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.149 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.159 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.194 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.215 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.239 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:45.249 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:45.257 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:45.267 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:43 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:45.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:45.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T17:03:45.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:03:45.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:45.332 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:45.340 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T17:03:45.353 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:45.363 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:45.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: Updating smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T17:03:45.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T17:03:45.450 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:03:45.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-14T17:03:45.494 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.507 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.541 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.555 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.563 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:45.582 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:45.609 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:45.622 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:43 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:45.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:45 smithi193 ceph-mon[35962]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-05-14T17:03:45.642 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:45 smithi150 ceph-mon[36204]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-05-14T17:03:45.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 -- 172.21.15.99:0/3692948474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98101b70 msgr2=0x7efe981076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:45.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 --2- 172.21.15.99:0/3692948474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98101b70 0x7efe981076a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009900 tx=0x7efe8c02a410 comp rx=0 tx=0).stop 2024-05-14T17:03:45.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 -- 172.21.15.99:0/3692948474 shutdown_connections 2024-05-14T17:03:45.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 --2- 172.21.15.99:0/3692948474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98107d80 0x7efe9810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 --2- 172.21.15.99:0/3692948474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe98101b70 0x7efe981076a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 --2- 172.21.15.99:0/3692948474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe981011c0 0x7efe981015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 -- 172.21.15.99:0/3692948474 >> 172.21.15.99:0/3692948474 conn(0x7efe98077ee0 msgr2=0x7efe980ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:45.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 -- 172.21.15.99:0/3692948474 shutdown_connections 2024-05-14T17:03:45.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.074+0000 7efe9dc23640 1 -- 172.21.15.99:0/3692948474 wait complete. 2024-05-14T17:03:45.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 Processor -- start 2024-05-14T17:03:45.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 -- start start 2024-05-14T17:03:45.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 0x7efe98102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe98107d80 0x7efe98102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98102bb0 0x7efe981a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe98112ce0 con 0x7efe981011c0 2024-05-14T17:03:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe98112b60 con 0x7efe98107d80 2024-05-14T17:03:45.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe9dc23640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe98112e60 con 0x7efe98102bb0 2024-05-14T17:03:45.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe977fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 0x7efe98102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe977fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 0x7efe98102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49156/0 (socket says 172.21.15.99:49156) 2024-05-14T17:03:46.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.075+0000 7efe977fe640 1 -- 172.21.15.99:0/1028993467 learned_addr learned my addr 172.21.15.99:0/1028993467 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe96ffd640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe98107d80 0x7efe98102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe97fff640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98102bb0 0x7efe981a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe977fe640 1 -- 172.21.15.99:0/1028993467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98102bb0 msgr2=0x7efe981a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe977fe640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98102bb0 0x7efe981a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe977fe640 1 -- 172.21.15.99:0/1028993467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe98107d80 msgr2=0x7efe98102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe977fe640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe98107d80 0x7efe98102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe977fe640 1 -- 172.21.15.99:0/1028993467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981a7a70 con 0x7efe981011c0 2024-05-14T17:03:46.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe97fff640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98102bb0 0x7efe981a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:46.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe96ffd640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe98107d80 0x7efe98102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:46.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe977fe640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 0x7efe98102130 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efe8800d770 tx=0x7efe8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:46.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88017070 con 0x7efe981011c0 2024-05-14T17:03:46.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe880054d0 con 0x7efe981011c0 2024-05-14T17:03:46.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981a7d60 con 0x7efe981011c0 2024-05-14T17:03:46.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88004c30 con 0x7efe981011c0 2024-05-14T17:03:46.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.076+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a81d0 con 0x7efe981011c0 2024-05-14T17:03:46.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.077+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe980507e0 con 0x7efe981011c0 2024-05-14T17:03:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.078+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7efe88004dd0 con 0x7efe981011c0 2024-05-14T17:03:46.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.079+0000 7efe94ff9640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efe800b2ac0 0x7efe800b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:46.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.079+0000 7efe96ffd640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efe800b2ac0 0x7efe800b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.079+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efe880e26d0 con 0x7efe981011c0 2024-05-14T17:03:46.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.080+0000 7efe96ffd640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efe800b2ac0 0x7efe800b4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe8c000c00 tx=0x7efe8c009610 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:46.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.083+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe880a9f70 con 0x7efe981011c0 2024-05-14T17:03:46.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:45 smithi099 ceph-mon[31851]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-05-14T17:03:46.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.273+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7efe98101b70 con 0x7efe981011c0 2024-05-14T17:03:46.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.277+0000 7efe94ff9640 1 -- 172.21.15.99:0/1028993467 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7efe880a9870 con 0x7efe981011c0 2024-05-14T17:03:46.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.279+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efe800b2ac0 msgr2=0x7efe800b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efe800b2ac0 0x7efe800b4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe8c000c00 tx=0x7efe8c009610 comp rx=0 tx=0).stop 2024-05-14T17:03:46.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 msgr2=0x7efe98102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 0x7efe98102130 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efe8800d770 tx=0x7efe8800dc30 comp rx=0 tx=0).stop 2024-05-14T17:03:46.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 shutdown_connections 2024-05-14T17:03:46.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efe800b2ac0 0x7efe800b4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe98102bb0 0x7efe981a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe98107d80 0x7efe98102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 --2- 172.21.15.99:0/1028993467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe981011c0 0x7efe98102130 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 >> 172.21.15.99:0/1028993467 conn(0x7efe98077ee0 msgr2=0x7efe98105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:46.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 shutdown_connections 2024-05-14T17:03:46.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:45.280+0000 7efe9dc23640 1 -- 172.21.15.99:0/1028993467 wait complete. 2024-05-14T17:03:46.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-14T17:03:46.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:46 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1028993467' entity='client.admin' 2024-05-14T17:03:46.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:46 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:46.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:46 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:47.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:46 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:47.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:46 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:47.090 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:46 smithi099 ceph-mon[31851]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-14T17:03:47.102 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:46 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1028993467' entity='client.admin' 2024-05-14T17:03:47.115 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:46 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:47.123 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:46 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:47.137 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:46 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:47.151 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:46 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:47.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:46 smithi150 ceph-mon[36204]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-14T17:03:47.177 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:46 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1028993467' entity='client.admin' 2024-05-14T17:03:47.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:46 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:47.217 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:46 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:47.233 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:46 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:47.243 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:46 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:47.259 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:46 smithi193 ceph-mon[35962]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-14T17:03:47.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 -- 172.21.15.99:0/1166318954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 msgr2=0x7fd0a4107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 --2- 172.21.15.99:0/1166318954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a4107e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009930 tx=0x7fd0a002a420 comp rx=0 tx=0).stop 2024-05-14T17:03:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 -- 172.21.15.99:0/1166318954 shutdown_connections 2024-05-14T17:03:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 --2- 172.21.15.99:0/1166318954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4101910 0x7fd0a410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 --2- 172.21.15.99:0/1166318954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a4100f50 0x7fd0a41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 --2- 172.21.15.99:0/1166318954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a4107e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 -- 172.21.15.99:0/1166318954 >> 172.21.15.99:0/1166318954 conn(0x7fd0a4077ee0 msgr2=0x7fd0a40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:47.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 -- 172.21.15.99:0/1166318954 shutdown_connections 2024-05-14T17:03:47.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.207+0000 7fd0ab346640 1 -- 172.21.15.99:0/1166318954 wait complete. 2024-05-14T17:03:47.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 Processor -- start 2024-05-14T17:03:47.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 -- start start 2024-05-14T17:03:47.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4100f50 0x7fd0a41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a4101910 0x7fd0a41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112df0 con 0x7fd0a4107a30 2024-05-14T17:03:47.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112c70 con 0x7fd0a4101910 2024-05-14T17:03:47.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0ab346640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4112f70 con 0x7fd0a4100f50 2024-05-14T17:03:47.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49186/0 (socket says 172.21.15.99:49186) 2024-05-14T17:03:47.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 -- 172.21.15.99:0/3094243964 learned_addr learned my addr 172.21.15.99:0/3094243964 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:47.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a88e1640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a4101910 0x7fd0a41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a90e2640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4100f50 0x7fd0a41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 -- 172.21.15.99:0/3094243964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4100f50 msgr2=0x7fd0a41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4100f50 0x7fd0a41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 -- 172.21.15.99:0/3094243964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a4101910 msgr2=0x7fd0a41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.208+0000 7fd0a98e3640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a4101910 0x7fd0a41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd0a98e3640 1 -- 172.21.15.99:0/3094243964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a419e2a0 con 0x7fd0a4107a30 2024-05-14T17:03:47.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd0a98e3640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a419db70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00eaa0 tx=0x7fd09c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:47.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c004d20 con 0x7fd0a4107a30 2024-05-14T17:03:47.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd09c005020 con 0x7fd0a4107a30 2024-05-14T17:03:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0116b0 con 0x7fd0a4107a30 2024-05-14T17:03:47.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a419e590 con 0x7fd0a4107a30 2024-05-14T17:03:47.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.209+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a5e20 con 0x7fd0a4107a30 2024-05-14T17:03:47.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.210+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a40507e0 con 0x7fd0a4107a30 2024-05-14T17:03:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.212+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fd09c0051d0 con 0x7fd0a4107a30 2024-05-14T17:03:47.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.213+0000 7fd09a7fc640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd0900b2b10 0x7fd0900b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.213+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fd09c0e2c40 con 0x7fd0a4107a30 2024-05-14T17:03:47.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.213+0000 7fd0a90e2640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd0900b2b10 0x7fd0900b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.214+0000 7fd0a90e2640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd0900b2b10 0x7fd0900b4fd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a0000bc0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:47.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.215+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd09c016030 con 0x7fd0a4107a30 2024-05-14T17:03:47.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.406+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd0a410af70 con 0x7fd0a4107a30 2024-05-14T17:03:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.409+0000 7fd09a7fc640 1 -- 172.21.15.99:0/3094243964 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fd09c0aa560 con 0x7fd0a4107a30 2024-05-14T17:03:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd0900b2b10 msgr2=0x7fd0900b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd0900b2b10 0x7fd0900b4fd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a0000bc0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 msgr2=0x7fd0a419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a419db70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00eaa0 tx=0x7fd09c00eea0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 shutdown_connections 2024-05-14T17:03:48.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd0900b2b10 0x7fd0900b4fd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0a4107a30 0x7fd0a419db70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a4101910 0x7fd0a41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 --2- 172.21.15.99:0/3094243964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4100f50 0x7fd0a41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 >> 172.21.15.99:0/3094243964 conn(0x7fd0a4077ee0 msgr2=0x7fd0a40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:48.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 shutdown_connections 2024-05-14T17:03:48.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:47.412+0000 7fd0ab346640 1 -- 172.21.15.99:0/3094243964 wait complete. 2024-05-14T17:03:48.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-14T17:03:48.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 -- 172.21.15.99:0/388765650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 msgr2=0x7f3900108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 --2- 172.21.15.99:0/388765650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900108e30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009930 tx=0x7f38f402a420 comp rx=0 tx=0).stop 2024-05-14T17:03:48.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 -- 172.21.15.99:0/388765650 shutdown_connections 2024-05-14T17:03:48.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 --2- 172.21.15.99:0/388765650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390006bbf0 0x7f390010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 --2- 172.21.15.99:0/388765650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f390006b230 0x7f390006b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 --2- 172.21.15.99:0/388765650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900108e30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 -- 172.21.15.99:0/388765650 >> 172.21.15.99:0/388765650 conn(0x7f3900078730 msgr2=0x7f3900078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:48.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 -- 172.21.15.99:0/388765650 shutdown_connections 2024-05-14T17:03:48.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.322+0000 7f39051f0640 1 -- 172.21.15.99:0/388765650 wait complete. 2024-05-14T17:03:48.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 Processor -- start 2024-05-14T17:03:48.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 -- start start 2024-05-14T17:03:48.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f390006b230 0x7f3900101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390006bbf0 0x7f3900102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3900112d90 con 0x7f3900108a50 2024-05-14T17:03:48.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3900112c10 con 0x7f390006bbf0 2024-05-14T17:03:48.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f39051f0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3900112f10 con 0x7f390006b230 2024-05-14T17:03:48.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f38ff59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f38ff59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900102780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49196/0 (socket says 172.21.15.99:49196) 2024-05-14T17:03:48.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f38ff59e640 1 -- 172.21.15.99:0/2061145269 learned_addr learned my addr 172.21.15.99:0/2061145269 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.323+0000 7f38fe59c640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390006bbf0 0x7f3900102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38fed9d640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f390006b230 0x7f3900101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38ff59e640 1 -- 172.21.15.99:0/2061145269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f390006b230 msgr2=0x7f3900101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38ff59e640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f390006b230 0x7f3900101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38ff59e640 1 -- 172.21.15.99:0/2061145269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390006bbf0 msgr2=0x7f3900102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38ff59e640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390006bbf0 0x7f3900102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38ff59e640 1 -- 172.21.15.99:0/2061145269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39001a77f0 con 0x7f3900108a50 2024-05-14T17:03:48.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38ff59e640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900102780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009fd0 tx=0x7f38f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8011830 con 0x7f3900108a50 2024-05-14T17:03:48.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39001a7ae0 con 0x7f3900108a50 2024-05-14T17:03:48.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f8016070 con 0x7f3900108a50 2024-05-14T17:03:48.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8010de0 con 0x7f3900108a50 2024-05-14T17:03:48.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.324+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39001a7fe0 con 0x7f3900108a50 2024-05-14T17:03:48.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.325+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3900104180 con 0x7f3900108a50 2024-05-14T17:03:48.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.327+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f38f80119d0 con 0x7f3900108a50 2024-05-14T17:03:48.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.328+0000 7f38e7fff640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f38e80b2ac0 0x7f38e80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:48.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.328+0000 7f38fed9d640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f38e80b2ac0 0x7f38e80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:48.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.328+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f38f80e23e0 con 0x7f3900108a50 2024-05-14T17:03:48.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.328+0000 7f38fed9d640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f38e80b2ac0 0x7f38e80b4f80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38f4000c00 tx=0x7f38f4031400 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:48.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.331+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38f80a9d90 con 0x7f3900108a50 2024-05-14T17:03:48.972 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:48 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3094243964' entity='client.admin' 2024-05-14T17:03:48.984 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:48 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:49.037 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:48 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:49.048 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:48 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:49.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:48 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:49.071 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:48 smithi193 ceph-mon[35962]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:49.083 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:48 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3094243964' entity='client.admin' 2024-05-14T17:03:49.093 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:48 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:49.108 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:48 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:49.176 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:48 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:49.185 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:48 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:49.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:48 smithi150 ceph-mon[36204]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:49.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:48 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3094243964' entity='client.admin' 2024-05-14T17:03:49.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:48 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:49.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:48 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:49.249 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:48 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:49.261 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:48 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:49.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:48 smithi099 ceph-mon[31851]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:49.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.517+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f390010aa20 con 0x7f3900108a50 2024-05-14T17:03:49.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.520+0000 7f38e7fff640 1 -- 172.21.15.99:0/2061145269 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f38f80a9690 con 0x7f3900108a50 2024-05-14T17:03:49.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f38e80b2ac0 msgr2=0x7f38e80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:49.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f38e80b2ac0 0x7f38e80b4f80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38f4000c00 tx=0x7f38f4031400 comp rx=0 tx=0).stop 2024-05-14T17:03:49.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 msgr2=0x7f3900102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:49.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900102780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009fd0 tx=0x7f38f800b8e0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 shutdown_connections 2024-05-14T17:03:49.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f38e80b2ac0 0x7f38e80b4f80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3900108a50 0x7f3900102780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390006bbf0 0x7f3900102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 --2- 172.21.15.99:0/2061145269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f390006b230 0x7f3900101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 >> 172.21.15.99:0/2061145269 conn(0x7f3900078730 msgr2=0x7f39000ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:49.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 shutdown_connections 2024-05-14T17:03:49.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:48.523+0000 7f39051f0640 1 -- 172.21.15.99:0/2061145269 wait complete. 2024-05-14T17:03:49.530 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-14T17:03:49.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 -- 172.21.15.99:0/1244870002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9740103c80 msgr2=0x7f9740104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:49.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 --2- 172.21.15.99:0/1244870002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9740103c80 0x7f9740104060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9730009900 tx=0x7f973002a480 comp rx=0 tx=0).stop 2024-05-14T17:03:49.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 -- 172.21.15.99:0/1244870002 shutdown_connections 2024-05-14T17:03:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 --2- 172.21.15.99:0/1244870002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f974010d050 0x7f974010f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 --2- 172.21.15.99:0/1244870002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97401045a0 0x7f974010ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 --2- 172.21.15.99:0/1244870002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9740103c80 0x7f9740104060 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 -- 172.21.15.99:0/1244870002 >> 172.21.15.99:0/1244870002 conn(0x7f9740078ad0 msgr2=0x7f9740078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:49.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 -- 172.21.15.99:0/1244870002 shutdown_connections 2024-05-14T17:03:49.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.646+0000 7f97465a4640 1 -- 172.21.15.99:0/1244870002 wait complete. 2024-05-14T17:03:49.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 Processor -- start 2024-05-14T17:03:49.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 -- start start 2024-05-14T17:03:49.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740103c80 0x7f9740199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 0x7f97401998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f974010d050 0x7f974019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:49.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9740112da0 con 0x7f97401045a0 2024-05-14T17:03:49.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9740112c20 con 0x7f974010d050 2024-05-14T17:03:49.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f97465a4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9740112f20 con 0x7f9740103c80 2024-05-14T17:03:49.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 0x7f97401998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:49.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 0x7f97401998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49224/0 (socket says 172.21.15.99:49224) 2024-05-14T17:03:49.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 -- 172.21.15.99:0/2425094163 learned_addr learned my addr 172.21.15.99:0/2425094163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:49.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973ffff640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740103c80 0x7f9740199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:49.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 -- 172.21.15.99:0/2425094163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740103c80 msgr2=0x7f9740199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740103c80 0x7f9740199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 -- 172.21.15.99:0/2425094163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f974010d050 msgr2=0x7f974019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:49.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f974010d050 0x7f974019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:49.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973f7fe640 1 -- 172.21.15.99:0/2425094163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974019e380 con 0x7f97401045a0 2024-05-14T17:03:49.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.647+0000 7f973ffff640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740103c80 0x7f9740199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:50.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.648+0000 7f973f7fe640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 0x7f97401998d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f973400b4a0 tx=0x7f973400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.648+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734014070 con 0x7f97401045a0 2024-05-14T17:03:50.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.648+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9734010ac0 con 0x7f97401045a0 2024-05-14T17:03:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.648+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974019e670 con 0x7f97401045a0 2024-05-14T17:03:50.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.648+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734017c30 con 0x7f97401045a0 2024-05-14T17:03:50.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.648+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97401a5f00 con 0x7f97401045a0 2024-05-14T17:03:50.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.649+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974006bb20 con 0x7f97401045a0 2024-05-14T17:03:50.127 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:49 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2061145269' entity='client.admin' 2024-05-14T17:03:50.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:50.144 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:50.175 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:50.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:50.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:49 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2061145269' entity='client.admin' 2024-05-14T17:03:50.200 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:50.209 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:50.217 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:50.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:50.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.650+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f9734017070 con 0x7f97401045a0 2024-05-14T17:03:50.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.651+0000 7f973d7fa640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f97280b2a90 0x7f97280b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:50.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.651+0000 7f973ffff640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f97280b2a90 0x7f97280b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:50.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.651+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f97340e1e00 con 0x7f97401045a0 2024-05-14T17:03:50.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.651+0000 7f973ffff640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f97280b2a90 0x7f97280b4f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9730000c00 tx=0x7f9730031400 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:50.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.654+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97340a97b0 con 0x7f97401045a0 2024-05-14T17:03:50.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.858+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9740062e90 con 0x7f97280b2a90 2024-05-14T17:03:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.926+0000 7f973d7fa640 1 -- 172.21.15.99:0/2425094163 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9740062e90 con 0x7f97280b2a90 2024-05-14T17:03:50.395 INFO:teuthology.orchestra.run.smithi099.stdout:Schedule set for path / 2024-05-14T17:03:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f97280b2a90 msgr2=0x7f97280b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:50.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f97280b2a90 0x7f97280b4f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9730000c00 tx=0x7f9730031400 comp rx=0 tx=0).stop 2024-05-14T17:03:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 msgr2=0x7f97401998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 0x7f97401998d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f973400b4a0 tx=0x7f973400b960 comp rx=0 tx=0).stop 2024-05-14T17:03:50.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 shutdown_connections 2024-05-14T17:03:50.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f97280b2a90 0x7f97280b4f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f974010d050 0x7f974019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97401045a0 0x7f97401998d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 --2- 172.21.15.99:0/2425094163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740103c80 0x7f9740199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.928+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 >> 172.21.15.99:0/2425094163 conn(0x7f9740078ad0 msgr2=0x7f97400ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:50.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.929+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 shutdown_connections 2024-05-14T17:03:50.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:49.929+0000 7f97465a4640 1 -- 172.21.15.99:0/2425094163 wait complete. 2024-05-14T17:03:50.635 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:49 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2061145269' entity='client.admin' 2024-05-14T17:03:50.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:03:50.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:03:50.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:03:50.674 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:03:50.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-14T17:03:50.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:50 smithi193 ceph-mon[35962]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-14T17:03:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:50 smithi193 ceph-mon[35962]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:50.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 -- 172.21.15.99:0/351542577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 msgr2=0x7fb2d4105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:50.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/351542577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d4105980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009900 tx=0x7fb2cc02a410 comp rx=0 tx=0).stop 2024-05-14T17:03:50.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 -- 172.21.15.99:0/351542577 shutdown_connections 2024-05-14T17:03:50.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/351542577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d4105980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/351542577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d4107df0 0x7fb2d406b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/351542577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4107980 0x7fb2d406b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 -- 172.21.15.99:0/351542577 >> 172.21.15.99:0/351542577 conn(0x7fb2d40786a0 msgr2=0x7fb2d4078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 -- 172.21.15.99:0/351542577 shutdown_connections 2024-05-14T17:03:50.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 -- 172.21.15.99:0/351542577 wait complete. 2024-05-14T17:03:50.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.858+0000 7fb2db6b0640 1 Processor -- start 2024-05-14T17:03:50.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 -- start start 2024-05-14T17:03:51.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:51.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d4107980 0x7fb2d419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:51.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4107df0 0x7fb2d41a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:51.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4119280 con 0x7fb2d41054d0 2024-05-14T17:03:51.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4119100 con 0x7fb2d4107df0 2024-05-14T17:03:51.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4119400 con 0x7fb2d4107980 2024-05-14T17:03:51.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49240/0 (socket says 172.21.15.99:49240) 2024-05-14T17:03:51.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 -- 172.21.15.99:0/3539367944 learned_addr learned my addr 172.21.15.99:0/3539367944 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:51.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d8c4b640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d4107980 0x7fb2d419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d9c4d640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4107df0 0x7fb2d41a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:51.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 -- 172.21.15.99:0/3539367944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d4107980 msgr2=0x7fb2d419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:51.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d4107980 0x7fb2d419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:51.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 -- 172.21.15.99:0/3539367944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4107df0 msgr2=0x7fb2d41a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:51.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4107df0 0x7fb2d41a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:51.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 -- 172.21.15.99:0/3539367944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d41a46f0 con 0x7fb2d41054d0 2024-05-14T17:03:51.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d944c640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d419f700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400ba50 tx=0x7fb2c400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:51.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4019070 con 0x7fb2d41054d0 2024-05-14T17:03:51.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c400cbe0 con 0x7fb2d41054d0 2024-05-14T17:03:51.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d41a49e0 con 0x7fb2d41054d0 2024-05-14T17:03:51.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.859+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40075d0 con 0x7fb2d41054d0 2024-05-14T17:03:51.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.860+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d41ac270 con 0x7fb2d41054d0 2024-05-14T17:03:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.861+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d4101fb0 con 0x7fb2d41054d0 2024-05-14T17:03:51.682 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:50 smithi150 ceph-mon[36204]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-14T17:03:51.689 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:50 smithi150 ceph-mon[36204]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:51.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.862+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fb2c401c070 con 0x7fb2d41054d0 2024-05-14T17:03:51.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.862+0000 7fb2d27fc640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2bc0b2b10 0x7fb2bc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:51.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.863+0000 7fb2d8c4b640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2bc0b2b10 0x7fb2bc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:51.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.863+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb2c40e1ca0 con 0x7fb2d41054d0 2024-05-14T17:03:51.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.863+0000 7fb2d8c4b640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2bc0b2b10 0x7fb2bc0b4fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8005fd0 tx=0x7fb2c8005e20 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:51.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:50.867+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2c40a9650 con 0x7fb2d41054d0 2024-05-14T17:03:51.852 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:50 smithi099 ceph-mon[31851]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-14T17:03:51.864 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:50 smithi099 ceph-mon[31851]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:51.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.073+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb2d4102fc0 con 0x7fb2bc0b2b10 2024-05-14T17:03:51.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.137+0000 7fb2d27fc640 1 -- 172.21.15.99:0/3539367944 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb2d4102fc0 con 0x7fb2bc0b2b10 2024-05-14T17:03:51.885 INFO:teuthology.orchestra.run.smithi099.stdout:Retention added to path / 2024-05-14T17:03:51.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:51 smithi193 ceph-mon[35962]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:51.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2bc0b2b10 msgr2=0x7fb2bc0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:51.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2bc0b2b10 0x7fb2bc0b4fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8005fd0 tx=0x7fb2c8005e20 comp rx=0 tx=0).stop 2024-05-14T17:03:51.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 msgr2=0x7fb2d419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:51.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d419f700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400ba50 tx=0x7fb2c400bf10 comp rx=0 tx=0).stop 2024-05-14T17:03:51.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 shutdown_connections 2024-05-14T17:03:51.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2bc0b2b10 0x7fb2bc0b4fd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:51.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4107df0 0x7fb2d41a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d4107980 0x7fb2d419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 --2- 172.21.15.99:0/3539367944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2d41054d0 0x7fb2d419f700 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 >> 172.21.15.99:0/3539367944 conn(0x7fb2d40786a0 msgr2=0x7fb2d41028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:52.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 shutdown_connections 2024-05-14T17:03:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:51.139+0000 7fb2db6b0640 1 -- 172.21.15.99:0/3539367944 wait complete. 2024-05-14T17:03:52.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T17:03:52.102 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:51 smithi099 ceph-mon[31851]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:52.111 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:51 smithi150 ceph-mon[36204]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:52.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 -- 172.21.15.99:0/4188297359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8101930 msgr2=0x7f60f8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:52.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 --2- 172.21.15.99:0/4188297359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8101930 0x7f60f8101db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009930 tx=0x7f60e802a420 comp rx=0 tx=0).stop 2024-05-14T17:03:52.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 -- 172.21.15.99:0/4188297359 shutdown_connections 2024-05-14T17:03:52.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 --2- 172.21.15.99:0/4188297359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60f81022f0 0x7f60f810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 --2- 172.21.15.99:0/4188297359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8101930 0x7f60f8101db0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 --2- 172.21.15.99:0/4188297359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f8107940 0x7f60f8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 -- 172.21.15.99:0/4188297359 >> 172.21.15.99:0/4188297359 conn(0x7f60f80fd420 msgr2=0x7f60f80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:52.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 -- 172.21.15.99:0/4188297359 shutdown_connections 2024-05-14T17:03:52.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 -- 172.21.15.99:0/4188297359 wait complete. 2024-05-14T17:03:52.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.217+0000 7f60fd938640 1 Processor -- start 2024-05-14T17:03:52.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 -- start start 2024-05-14T17:03:52.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f8101930 0x7f60f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:52.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60f81022f0 0x7f60f8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:52.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 0x7f60f819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:52.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8111c00 con 0x7f60f8107940 2024-05-14T17:03:52.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8111a80 con 0x7f60f8101930 2024-05-14T17:03:52.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8111d80 con 0x7f60f81022f0 2024-05-14T17:03:52.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 0x7f60f819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:52.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 0x7f60f819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49262/0 (socket says 172.21.15.99:49262) 2024-05-14T17:03:52.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 -- 172.21.15.99:0/276994859 learned_addr learned my addr 172.21.15.99:0/276994859 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:52.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f67fc640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60f81022f0 0x7f60f8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:52.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f6ffd640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f8101930 0x7f60f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:52.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 -- 172.21.15.99:0/276994859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60f81022f0 msgr2=0x7f60f8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:52.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60f81022f0 0x7f60f8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 -- 172.21.15.99:0/276994859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f8101930 msgr2=0x7f60f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:52.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f8101930 0x7f60f81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 -- 172.21.15.99:0/276994859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f819d0d0 con 0x7f60f8107940 2024-05-14T17:03:52.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60f77fe640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 0x7f60f819c9a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c970 tx=0x7f60f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:52.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0007bb0 con 0x7f60f8107940 2024-05-14T17:03:52.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f0007d50 con 0x7f60f8107940 2024-05-14T17:03:52.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f819d3c0 con 0x7f60f8107940 2024-05-14T17:03:52.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.218+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0004e60 con 0x7f60f8107940 2024-05-14T17:03:52.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.219+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81a4c50 con 0x7f60f8107940 2024-05-14T17:03:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.219+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f8103070 con 0x7f60f8107940 2024-05-14T17:03:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.221+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f60f0005000 con 0x7f60f8107940 2024-05-14T17:03:52.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.222+0000 7f60fc936640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f60e00b2a90 0x7f60e00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.222+0000 7f60f6ffd640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f60e00b2a90 0x7f60e00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:52.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.222+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f60f00e2910 con 0x7f60f8107940 2024-05-14T17:03:52.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.222+0000 7f60f6ffd640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f60e00b2a90 0x7f60e00b4f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60ec005fd0 tx=0x7f60ec009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:52.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.226+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60f0014030 con 0x7f60f8107940 2024-05-14T17:03:52.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.428+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f60f8062e90 con 0x7f60e00b2a90 2024-05-14T17:03:52.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.436+0000 7f60fc936640 1 -- 172.21.15.99:0/276994859 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f60f8062e90 con 0x7f60e00b2a90 2024-05-14T17:03:52.751 INFO:teuthology.orchestra.run.smithi099.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T17:03:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-14T17:03:52.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f60e00b2a90 msgr2=0x7f60e00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:52.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f60e00b2a90 0x7f60e00b4f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60ec005fd0 tx=0x7f60ec009260 comp rx=0 tx=0).stop 2024-05-14T17:03:52.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 msgr2=0x7f60f819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:52.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 0x7f60f819c9a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c970 tx=0x7f60f000ce30 comp rx=0 tx=0).stop 2024-05-14T17:03:52.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 shutdown_connections 2024-05-14T17:03:52.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f60e00b2a90 0x7f60e00b4f50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f8107940 0x7f60f819c9a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60f81022f0 0x7f60f8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 --2- 172.21.15.99:0/276994859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f8101930 0x7f60f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:52.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 >> 172.21.15.99:0/276994859 conn(0x7f60f80fd420 msgr2=0x7f60f80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:52.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 shutdown_connections 2024-05-14T17:03:52.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:52.438+0000 7f60fd938640 1 -- 172.21.15.99:0/276994859 wait complete. 2024-05-14T17:03:52.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T17:03:52.937 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:52 smithi150 ceph-mon[36204]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:52.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:52 smithi193 ceph-mon[35962]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:52.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:52 smithi099 ceph-mon[31851]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:53.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 -- 172.21.15.99:0/1450106206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 msgr2=0x7f6938107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:53.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 --2- 172.21.15.99:0/1450106206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f6938107d20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6928009930 tx=0x7f692802a420 comp rx=0 tx=0).stop 2024-05-14T17:03:53.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 -- 172.21.15.99:0/1450106206 shutdown_connections 2024-05-14T17:03:53.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 --2- 172.21.15.99:0/1450106206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6938102340 0x7f693810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 --2- 172.21.15.99:0/1450106206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938101980 0x7f6938101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 --2- 172.21.15.99:0/1450106206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f6938107d20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 -- 172.21.15.99:0/1450106206 >> 172.21.15.99:0/1450106206 conn(0x7f6938078ad0 msgr2=0x7f6938078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:53.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 -- 172.21.15.99:0/1450106206 shutdown_connections 2024-05-14T17:03:53.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.064+0000 7f693dc83640 1 -- 172.21.15.99:0/1450106206 wait complete. 2024-05-14T17:03:53.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 Processor -- start 2024-05-14T17:03:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 -- start start 2024-05-14T17:03:53.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6938101980 0x7f6938198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938102340 0x7f69381985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f693819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111bd0 con 0x7f6938107940 2024-05-14T17:03:53.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111a50 con 0x7f6938102340 2024-05-14T17:03:53.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f693dc83640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111d50 con 0x7f6938101980 2024-05-14T17:03:53.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f693819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f693819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49278/0 (socket says 172.21.15.99:49278) 2024-05-14T17:03:53.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 -- 172.21.15.99:0/889109299 learned_addr learned my addr 172.21.15.99:0/889109299 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:53.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6936ffd640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938102340 0x7f69381985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 -- 172.21.15.99:0/889109299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6938101980 msgr2=0x7f6938198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:53.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f69377fe640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6938101980 0x7f6938198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6938101980 0x7f6938198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 -- 172.21.15.99:0/889109299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938102340 msgr2=0x7f69381985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:53.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938102340 0x7f69381985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6937fff640 1 -- 172.21.15.99:0/889109299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693819d060 con 0x7f6938107940 2024-05-14T17:03:53.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.065+0000 7f6936ffd640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938102340 0x7f69381985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:53.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.066+0000 7f6937fff640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f693819c930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6930007cb0 tx=0x7f693000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:53.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.066+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693000eb30 con 0x7f6938107940 2024-05-14T17:03:53.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.066+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693000ecd0 con 0x7f6938107940 2024-05-14T17:03:53.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.066+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693819d350 con 0x7f6938107940 2024-05-14T17:03:53.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.066+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69300153f0 con 0x7f6938107940 2024-05-14T17:03:53.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.066+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693806a100 con 0x7f6938107940 2024-05-14T17:03:53.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.067+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69380507e0 con 0x7f6938107940 2024-05-14T17:03:53.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.068+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f693000a440 con 0x7f6938107940 2024-05-14T17:03:53.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.068+0000 7f6934ff9640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f69200b2a90 0x7f69200b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:53.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.069+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f69300e43b0 con 0x7f6938107940 2024-05-14T17:03:53.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.069+0000 7f69377fe640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f69200b2a90 0x7f69200b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:53.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.069+0000 7f69377fe640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f69200b2a90 0x7f69200b4f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6928006010 tx=0x7f6928000bc0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:53.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.073+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69300abc50 con 0x7f6938107940 2024-05-14T17:03:53.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.275+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f693806a360 con 0x7f69200b2a90 2024-05-14T17:03:53.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.283+0000 7f6934ff9640 1 -- 172.21.15.99:0/889109299 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f693806a360 con 0x7f69200b2a90 2024-05-14T17:03:53.688 INFO:teuthology.orchestra.run.smithi099.stdout:/ 1m 6m3h 2024-05-14T17:03:53.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f69200b2a90 msgr2=0x7f69200b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:53.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f69200b2a90 0x7f69200b4f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6928006010 tx=0x7f6928000bc0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 msgr2=0x7f693819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f693819c930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6930007cb0 tx=0x7f693000e420 comp rx=0 tx=0).stop 2024-05-14T17:03:53.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 shutdown_connections 2024-05-14T17:03:53.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f69200b2a90 0x7f69200b4f50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6938107940 0x7f693819c930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6938102340 0x7f69381985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 --2- 172.21.15.99:0/889109299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6938101980 0x7f6938198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:53.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 >> 172.21.15.99:0/889109299 conn(0x7f6938078ad0 msgr2=0x7f6938104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:53.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 shutdown_connections 2024-05-14T17:03:53.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:53.285+0000 7f693dc83640 1 -- 172.21.15.99:0/889109299 wait complete. 2024-05-14T17:03:53.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-14T17:03:53.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:53 smithi193 ceph-mon[35962]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:53.929 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:53 smithi150 ceph-mon[36204]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:53.936 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:53 smithi099 ceph-mon[31851]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:53.971 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-14T17:03:53.988 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-14T17:03:54.062 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:03:54.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-14T17:03:54.094 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-14T17:03:54.111 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T17:03:54.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:03:54.363 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:54.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:54 smithi099 ceph-mon[31851]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:54.770 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:54 smithi099 ceph-mon[31851]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:54.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:54 smithi193 ceph-mon[35962]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:54.799 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:54 smithi193 ceph-mon[35962]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:54.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:54 smithi150 ceph-mon[36204]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:03:54.976 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:54 smithi150 ceph-mon[36204]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-14T17:03:55.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/2983094689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb941095a0 msgr2=0x7fbb94109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:55.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/2983094689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb941095a0 0x7fbb94109980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009990 tx=0x7fbb84009ca0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/2983094689 shutdown_connections 2024-05-14T17:03:56.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/2983094689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406c470 0x7fbb941103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/2983094689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb941095a0 0x7fbb94109980 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/2983094689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9406bab0 0x7fbb9406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/2983094689 >> 172.21.15.99:0/2983094689 conn(0x7fbb940793b0 msgr2=0x7fbb940797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:56.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.972+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/2983094689 shutdown_connections 2024-05-14T17:03:56.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.973+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/2983094689 wait complete. 2024-05-14T17:03:56.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.973+0000 7fbb9a9bc700 1 Processor -- start 2024-05-14T17:03:56.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.973+0000 7fbb9a9bc700 1 -- start start 2024-05-14T17:03:56.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb9a9bc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406bab0 0x7fbb9419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:56.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb9a9bc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 0x7fbb9419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:56.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb9a9bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb941095a0 0x7fbb94199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb9a9bc700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94113730 con 0x7fbb9406c470 2024-05-14T17:03:56.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb9a9bc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941135b0 con 0x7fbb9406bab0 2024-05-14T17:03:56.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb9a9bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941138b0 con 0x7fbb941095a0 2024-05-14T17:03:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 0x7fbb9419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:56.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 0x7fbb9419fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44628/0 (socket says 172.21.15.99:44628) 2024-05-14T17:03:56.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 -- 172.21.15.99:0/710602742 learned_addr learned my addr 172.21.15.99:0/710602742 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:56.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb93fff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406bab0 0x7fbb9419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb98f59700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb941095a0 0x7fbb94199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:56.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 -- 172.21.15.99:0/710602742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb941095a0 msgr2=0x7fbb94199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:56.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb941095a0 0x7fbb94199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 -- 172.21.15.99:0/710602742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406bab0 msgr2=0x7fbb9419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:56.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406bab0 0x7fbb9419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 -- 172.21.15.99:0/710602742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9419a230 con 0x7fbb9406c470 2024-05-14T17:03:56.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb93fff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406bab0 0x7fbb9419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:56.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.974+0000 7fbb8bfff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 0x7fbb9419fe50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00d950 tx=0x7fbb7c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:56.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.975+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c016070 con 0x7fbb9406c470 2024-05-14T17:03:56.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.975+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb7c010460 con 0x7fbb9406c470 2024-05-14T17:03:56.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.975+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9419a520 con 0x7fbb9406c470 2024-05-14T17:03:56.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.975+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00f620 con 0x7fbb9406c470 2024-05-14T17:03:56.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.975+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a67e0 con 0x7fbb9406c470 2024-05-14T17:03:56.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.976+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9410ba80 con 0x7fbb9406c470 2024-05-14T17:03:56.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.978+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fbb7c019070 con 0x7fbb9406c470 2024-05-14T17:03:56.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.979+0000 7fbb91ffb700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb800b2b00 0x7fbb800b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.979+0000 7fbb93fff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb800b2b00 0x7fbb800b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.979+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fbb7c0e23b0 con 0x7fbb9406c470 2024-05-14T17:03:56.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.979+0000 7fbb93fff700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb800b2b00 0x7fbb800b4fc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb8400b540 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:55.982+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb7c0a9c50 con 0x7fbb9406c470 2024-05-14T17:03:56.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.185+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb94063980 con 0x7fbb800b2b00 2024-05-14T17:03:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.193+0000 7fbb91ffb700 1 -- 172.21.15.99:0/710602742 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257762 (secure 0 0 0) 0x7fbb94063980 con 0x7fbb800b2b00 2024-05-14T17:03:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:03:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb800b2b00 msgr2=0x7fbb800b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:56.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb800b2b00 0x7fbb800b4fc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb8400b540 comp rx=0 tx=0).stop 2024-05-14T17:03:56.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 msgr2=0x7fbb9419fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:56.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 0x7fbb9419fe50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00d950 tx=0x7fbb7c00dd10 comp rx=0 tx=0).stop 2024-05-14T17:03:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 shutdown_connections 2024-05-14T17:03:56.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb800b2b00 0x7fbb800b4fc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb941095a0 0x7fbb94199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9406bab0 0x7fbb9419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 --2- 172.21.15.99:0/710602742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9406c470 0x7fbb9419fe50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:56.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 >> 172.21.15.99:0/710602742 conn(0x7fbb940793b0 msgr2=0x7fbb941011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:56.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 shutdown_connections 2024-05-14T17:03:56.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:56.196+0000 7fbb9a9bc700 1 -- 172.21.15.99:0/710602742 wait complete. 2024-05-14T17:03:56.888 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:03:56.897 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-05-14T17:03:55.593105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384424,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1756,"num_read_kb":1267,"num_write":554,"num_write_kb":4503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2952887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":907,"ondisk_log_size":907,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372684,"kb_used_data":14832,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569140,"statfs":{"total":1151940427776,"available":1151558799360,"internally_reserved":0,"allocated":15187968,"data_stored":9445800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":401,"num_read_kb":449,"num_write":138,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002776"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687318+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:03:37.687318+0000","last_peered":"2024-05-14T17:03:37.687318+0000","last_clean":"2024-05-14T17:03:37.687318+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:03:37.687318+0000","last_undegraded":"2024-05-14T17:03:37.687318+0000","last_fullsized":"2024-05-14T17:03:37.687318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687622+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:03:37.687622+0000","last_peered":"2024-05-14T17:03:37.687622+0000","last_clean":"2024-05-14T17:03:37.687622+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:03:37.687622+0000","last_undegraded":"2024-05-14T17:03:37.687622+0000","last_fullsized":"2024-05-14T17:03:37.687622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687406+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:03:37.687406+0000","last_peered":"2024-05-14T17:03:37.687406+0000","last_clean":"2024-05-14T17:03:37.687406+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:03:37.687406+0000","last_undegraded":"2024-05-14T17:03:37.687406+0000","last_fullsized":"2024-05-14T17:03:37.687406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577663+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:03:37.577663+0000","last_peered":"2024-05-14T17:03:37.577663+0000","last_clean":"2024-05-14T17:03:37.577663+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:03:37.577663+0000","last_undegraded":"2024-05-14T17:03:37.577663+0000","last_fullsized":"2024-05-14T17:03:37.577663+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674251+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:03:37.674251+0000","last_peered":"2024-05-14T17:03:37.674251+0000","last_clean":"2024-05-14T17:03:37.674251+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:03:37.674251+0000","last_undegraded":"2024-05-14T17:03:37.674251+0000","last_fullsized":"2024-05-14T17:03:37.674251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683335+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:03:37.683335+0000","last_peered":"2024-05-14T17:03:37.683335+0000","last_clean":"2024-05-14T17:03:37.683335+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:03:37.683335+0000","last_undegraded":"2024-05-14T17:03:37.683335+0000","last_fullsized":"2024-05-14T17:03:37.683335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789151+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:03:37.789151+0000","last_peered":"2024-05-14T17:03:37.789151+0000","last_clean":"2024-05-14T17:03:37.789151+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:03:37.789151+0000","last_undegraded":"2024-05-14T17:03:37.789151+0000","last_fullsized":"2024-05-14T17:03:37.789151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.673941+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:03:37.673941+0000","last_peered":"2024-05-14T17:03:37.673941+0000","last_clean":"2024-05-14T17:03:37.673941+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:03:37.673941+0000","last_undegraded":"2024-05-14T17:03:37.673941+0000","last_fullsized":"2024-05-14T17:03:37.673941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687162+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:03:37.687162+0000","last_peered":"2024-05-14T17:03:37.687162+0000","last_clean":"2024-05-14T17:03:37.687162+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:03:37.687162+0000","last_undegraded":"2024-05-14T17:03:37.687162+0000","last_fullsized":"2024-05-14T17:03:37.687162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683580+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:03:37.683580+0000","last_peered":"2024-05-14T17:03:37.683580+0000","last_clean":"2024-05-14T17:03:37.683580+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:03:37.683580+0000","last_undegraded":"2024-05-14T17:03:37.683580+0000","last_fullsized":"2024-05-14T17:03:37.683580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789385+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:03:37.789385+0000","last_peered":"2024-05-14T17:03:37.789385+0000","last_clean":"2024-05-14T17:03:37.789385+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:03:37.789385+0000","last_undegraded":"2024-05-14T17:03:37.789385+0000","last_fullsized":"2024-05-14T17:03:37.789385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577660+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:03:37.577660+0000","last_peered":"2024-05-14T17:03:37.577660+0000","last_clean":"2024-05-14T17:03:37.577660+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:03:37.577660+0000","last_undegraded":"2024-05-14T17:03:37.577660+0000","last_fullsized":"2024-05-14T17:03:37.577660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688782+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:03:37.688782+0000","last_peered":"2024-05-14T17:03:37.688782+0000","last_clean":"2024-05-14T17:03:37.688782+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:03:37.688782+0000","last_undegraded":"2024-05-14T17:03:37.688782+0000","last_fullsized":"2024-05-14T17:03:37.688782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686967+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:03:37.686967+0000","last_peered":"2024-05-14T17:03:37.686967+0000","last_clean":"2024-05-14T17:03:37.686967+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:03:37.686967+0000","last_undegraded":"2024-05-14T17:03:37.686967+0000","last_fullsized":"2024-05-14T17:03:37.686967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686877+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:03:37.686877+0000","last_peered":"2024-05-14T17:03:37.686877+0000","last_clean":"2024-05-14T17:03:37.686877+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:03:37.686877+0000","last_undegraded":"2024-05-14T17:03:37.686877+0000","last_fullsized":"2024-05-14T17:03:37.686877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686703+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:03:37.686703+0000","last_peered":"2024-05-14T17:03:37.686703+0000","last_clean":"2024-05-14T17:03:37.686703+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:03:37.686703+0000","last_undegraded":"2024-05-14T17:03:37.686703+0000","last_fullsized":"2024-05-14T17:03:37.686703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687437+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:03:37.687437+0000","last_peered":"2024-05-14T17:03:37.687437+0000","last_clean":"2024-05-14T17:03:37.687437+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:03:37.687437+0000","last_undegraded":"2024-05-14T17:03:37.687437+0000","last_fullsized":"2024-05-14T17:03:37.687437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573578+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:03:37.573578+0000","last_peered":"2024-05-14T17:03:37.573578+0000","last_clean":"2024-05-14T17:03:37.573578+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:03:37.573578+0000","last_undegraded":"2024-05-14T17:03:37.573578+0000","last_fullsized":"2024-05-14T17:03:37.573578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575796+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:03:37.575796+0000","last_peered":"2024-05-14T17:03:37.575796+0000","last_clean":"2024-05-14T17:03:37.575796+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:03:37.575796+0000","last_undegraded":"2024-05-14T17:03:37.575796+0000","last_fullsized":"2024-05-14T17:03:37.575796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573421+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:03:37.573421+0000","last_peered":"2024-05-14T17:03:37.573421+0000","last_clean":"2024-05-14T17:03:37.573421+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:03:37.573421+0000","last_undegraded":"2024-05-14T17:03:37.573421+0000","last_fullsized":"2024-05-14T17:03:37.573421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674758+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:03:37.674758+0000","last_peered":"2024-05-14T17:03:37.674758+0000","last_clean":"2024-05-14T17:03:37.674758+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:03:37.674758+0000","last_undegraded":"2024-05-14T17:03:37.674758+0000","last_fullsized":"2024-05-14T17:03:37.674758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674912+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:03:37.674912+0000","last_peered":"2024-05-14T17:03:37.674912+0000","last_clean":"2024-05-14T17:03:37.674912+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:03:37.674912+0000","last_undegraded":"2024-05-14T17:03:37.674912+0000","last_fullsized":"2024-05-14T17:03:37.674912+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687882+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:03:37.687882+0000","last_peered":"2024-05-14T17:03:37.687882+0000","last_clean":"2024-05-14T17:03:37.687882+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:03:37.687882+0000","last_undegraded":"2024-05-14T17:03:37.687882+0000","last_fullsized":"2024-05-14T17:03:37.687882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674597+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:03:37.674597+0000","last_peered":"2024-05-14T17:03:37.674597+0000","last_clean":"2024-05-14T17:03:37.674597+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:03:37.674597+0000","last_undegraded":"2024-05-14T17:03:37.674597+0000","last_fullsized":"2024-05-14T17:03:37.674597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686859+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:03:37.686859+0000","last_peered":"2024-05-14T17:03:37.686859+0000","last_clean":"2024-05-14T17:03:37.686859+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:03:37.686859+0000","last_undegraded":"2024-05-14T17:03:37.686859+0000","last_fullsized":"2024-05-14T17:03:37.686859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674465+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:03:37.674465+0000","last_peered":"2024-05-14T17:03:37.674465+0000","last_clean":"2024-05-14T17:03:37.674465+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:03:37.674465+0000","last_undegraded":"2024-05-14T17:03:37.674465+0000","last_fullsized":"2024-05-14T17:03:37.674465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680651+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:03:37.680651+0000","last_peered":"2024-05-14T17:03:37.680651+0000","last_clean":"2024-05-14T17:03:37.680651+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:03:37.680651+0000","last_undegraded":"2024-05-14T17:03:37.680651+0000","last_fullsized":"2024-05-14T17:03:37.680651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":261,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.541061+0000","last_change":"2024-05-14T17:01:04.271215+0000","last_active":"2024-05-14T17:03:52.541061+0000","last_peered":"2024-05-14T17:03:52.541061+0000","last_clean":"2024-05-14T17:03:52.541061+0000","last_became_active":"2024-05-14T17:01:04.267186+0000","last_became_peered":"2024-05-14T17:01:04.267186+0000","last_unstale":"2024-05-14T17:03:52.541061+0000","last_undegraded":"2024-05-14T17:03:52.541061+0000","last_fullsized":"2024-05-14T17:03:52.541061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:01.465730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674220+0000","last_change":"2024-05-14T17:01:04.241079+0000","last_active":"2024-05-14T17:03:37.674220+0000","last_peered":"2024-05-14T17:03:37.674220+0000","last_clean":"2024-05-14T17:03:37.674220+0000","last_became_active":"2024-05-14T17:01:04.240698+0000","last_became_peered":"2024-05-14T17:01:04.240698+0000","last_unstale":"2024-05-14T17:03:37.674220+0000","last_undegraded":"2024-05-14T17:03:37.674220+0000","last_fullsized":"2024-05-14T17:03:37.674220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:10.589991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683244+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:03:37.683244+0000","last_peered":"2024-05-14T17:03:37.683244+0000","last_clean":"2024-05-14T17:03:37.683244+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:03:37.683244+0000","last_undegraded":"2024-05-14T17:03:37.683244+0000","last_fullsized":"2024-05-14T17:03:37.683244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680431+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:03:37.680431+0000","last_peered":"2024-05-14T17:03:37.680431+0000","last_clean":"2024-05-14T17:03:37.680431+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:03:37.680431+0000","last_undegraded":"2024-05-14T17:03:37.680431+0000","last_fullsized":"2024-05-14T17:03:37.680431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687440+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:03:37.687440+0000","last_peered":"2024-05-14T17:03:37.687440+0000","last_clean":"2024-05-14T17:03:37.687440+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:03:37.687440+0000","last_undegraded":"2024-05-14T17:03:37.687440+0000","last_fullsized":"2024-05-14T17:03:37.687440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573316+0000","last_change":"2024-05-14T17:01:04.236554+0000","last_active":"2024-05-14T17:03:37.573316+0000","last_peered":"2024-05-14T17:03:37.573316+0000","last_clean":"2024-05-14T17:03:37.573316+0000","last_became_active":"2024-05-14T17:01:04.236203+0000","last_became_peered":"2024-05-14T17:01:04.236203+0000","last_unstale":"2024-05-14T17:03:37.573316+0000","last_undegraded":"2024-05-14T17:03:37.573316+0000","last_fullsized":"2024-05-14T17:03:37.573316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:34.421246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680247+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:03:37.680247+0000","last_peered":"2024-05-14T17:03:37.680247+0000","last_clean":"2024-05-14T17:03:37.680247+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:03:37.680247+0000","last_undegraded":"2024-05-14T17:03:37.680247+0000","last_fullsized":"2024-05-14T17:03:37.680247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573284+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:03:37.573284+0000","last_peered":"2024-05-14T17:03:37.573284+0000","last_clean":"2024-05-14T17:03:37.573284+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:03:37.573284+0000","last_undegraded":"2024-05-14T17:03:37.573284+0000","last_fullsized":"2024-05-14T17:03:37.573284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":240,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.540607+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:03:52.540607+0000","last_peered":"2024-05-14T17:03:52.540607+0000","last_clean":"2024-05-14T17:03:52.540607+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:03:52.540607+0000","last_undegraded":"2024-05-14T17:03:52.540607+0000","last_fullsized":"2024-05-14T17:03:52.540607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687113+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:03:37.687113+0000","last_peered":"2024-05-14T17:03:37.687113+0000","last_clean":"2024-05-14T17:03:37.687113+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:03:37.687113+0000","last_undegraded":"2024-05-14T17:03:37.687113+0000","last_fullsized":"2024-05-14T17:03:37.687113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789349+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:03:37.789349+0000","last_peered":"2024-05-14T17:03:37.789349+0000","last_clean":"2024-05-14T17:03:37.789349+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:03:37.789349+0000","last_undegraded":"2024-05-14T17:03:37.789349+0000","last_fullsized":"2024-05-14T17:03:37.789349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687808+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:03:37.687808+0000","last_peered":"2024-05-14T17:03:37.687808+0000","last_clean":"2024-05-14T17:03:37.687808+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:03:37.687808+0000","last_undegraded":"2024-05-14T17:03:37.687808+0000","last_fullsized":"2024-05-14T17:03:37.687808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680791+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:03:37.680791+0000","last_peered":"2024-05-14T17:03:37.680791+0000","last_clean":"2024-05-14T17:03:37.680791+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:03:37.680791+0000","last_undegraded":"2024-05-14T17:03:37.680791+0000","last_fullsized":"2024-05-14T17:03:37.680791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686928+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:03:37.686928+0000","last_peered":"2024-05-14T17:03:37.686928+0000","last_clean":"2024-05-14T17:03:37.686928+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:03:37.686928+0000","last_undegraded":"2024-05-14T17:03:37.686928+0000","last_fullsized":"2024-05-14T17:03:37.686928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":488,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:53.541533+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:03:53.541533+0000","last_peered":"2024-05-14T17:03:53.541533+0000","last_clean":"2024-05-14T17:03:53.541533+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:03:53.541533+0000","last_undegraded":"2024-05-14T17:03:53.541533+0000","last_fullsized":"2024-05-14T17:03:53.541533+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":424,"num_read_kb":278,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674413+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:03:37.674413+0000","last_peered":"2024-05-14T17:03:37.674413+0000","last_clean":"2024-05-14T17:03:37.674413+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:03:37.674413+0000","last_undegraded":"2024-05-14T17:03:37.674413+0000","last_fullsized":"2024-05-14T17:03:37.674413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687117+0000","last_change":"2024-05-14T17:01:04.249537+0000","last_active":"2024-05-14T17:03:37.687117+0000","last_peered":"2024-05-14T17:03:37.687117+0000","last_clean":"2024-05-14T17:03:37.687117+0000","last_became_active":"2024-05-14T17:01:04.239963+0000","last_became_peered":"2024-05-14T17:01:04.239963+0000","last_unstale":"2024-05-14T17:03:37.687117+0000","last_undegraded":"2024-05-14T17:03:37.687117+0000","last_fullsized":"2024-05-14T17:03:37.687117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:00:51.798116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688287+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:03:37.688287+0000","last_peered":"2024-05-14T17:03:37.688287+0000","last_clean":"2024-05-14T17:03:37.688287+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:03:37.688287+0000","last_undegraded":"2024-05-14T17:03:37.688287+0000","last_fullsized":"2024-05-14T17:03:37.688287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680371+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:03:37.680371+0000","last_peered":"2024-05-14T17:03:37.680371+0000","last_clean":"2024-05-14T17:03:37.680371+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:03:37.680371+0000","last_undegraded":"2024-05-14T17:03:37.680371+0000","last_fullsized":"2024-05-14T17:03:37.680371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789348+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:03:37.789348+0000","last_peered":"2024-05-14T17:03:37.789348+0000","last_clean":"2024-05-14T17:03:37.789348+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:03:37.789348+0000","last_undegraded":"2024-05-14T17:03:37.789348+0000","last_fullsized":"2024-05-14T17:03:37.789348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.448350+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:03:52.448350+0000","last_peered":"2024-05-14T17:03:52.448350+0000","last_clean":"2024-05-14T17:03:52.448350+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:03:52.448350+0000","last_undegraded":"2024-05-14T17:03:52.448350+0000","last_fullsized":"2024-05-14T17:03:52.448350+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789088+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:03:37.789088+0000","last_peered":"2024-05-14T17:03:37.789088+0000","last_clean":"2024-05-14T17:03:37.789088+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:03:37.789088+0000","last_undegraded":"2024-05-14T17:03:37.789088+0000","last_fullsized":"2024-05-14T17:03:37.789088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674198+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:03:37.674198+0000","last_peered":"2024-05-14T17:03:37.674198+0000","last_clean":"2024-05-14T17:03:37.674198+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:03:37.674198+0000","last_undegraded":"2024-05-14T17:03:37.674198+0000","last_fullsized":"2024-05-14T17:03:37.674198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680298+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:03:37.680298+0000","last_peered":"2024-05-14T17:03:37.680298+0000","last_clean":"2024-05-14T17:03:37.680298+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:03:37.680298+0000","last_undegraded":"2024-05-14T17:03:37.680298+0000","last_fullsized":"2024-05-14T17:03:37.680298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674493+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:03:37.674493+0000","last_peered":"2024-05-14T17:03:37.674493+0000","last_clean":"2024-05-14T17:03:37.674493+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:03:37.674493+0000","last_undegraded":"2024-05-14T17:03:37.674493+0000","last_fullsized":"2024-05-14T17:03:37.674493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687257+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:03:37.687257+0000","last_peered":"2024-05-14T17:03:37.687257+0000","last_clean":"2024-05-14T17:03:37.687257+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:03:37.687257+0000","last_undegraded":"2024-05-14T17:03:37.687257+0000","last_fullsized":"2024-05-14T17:03:37.687257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573679+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:03:37.573679+0000","last_peered":"2024-05-14T17:03:37.573679+0000","last_clean":"2024-05-14T17:03:37.573679+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:03:37.573679+0000","last_undegraded":"2024-05-14T17:03:37.573679+0000","last_fullsized":"2024-05-14T17:03:37.573679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683414+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:03:37.683414+0000","last_peered":"2024-05-14T17:03:37.683414+0000","last_clean":"2024-05-14T17:03:37.683414+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:03:37.683414+0000","last_undegraded":"2024-05-14T17:03:37.683414+0000","last_fullsized":"2024-05-14T17:03:37.683414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789470+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:03:37.789470+0000","last_peered":"2024-05-14T17:03:37.789470+0000","last_clean":"2024-05-14T17:03:37.789470+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:03:37.789470+0000","last_undegraded":"2024-05-14T17:03:37.789470+0000","last_fullsized":"2024-05-14T17:03:37.789470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683335+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:03:37.683335+0000","last_peered":"2024-05-14T17:03:37.683335+0000","last_clean":"2024-05-14T17:03:37.683335+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:03:37.683335+0000","last_undegraded":"2024-05-14T17:03:37.683335+0000","last_fullsized":"2024-05-14T17:03:37.683335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573215+0000","last_change":"2024-05-14T17:01:04.237414+0000","last_active":"2024-05-14T17:03:37.573215+0000","last_peered":"2024-05-14T17:03:37.573215+0000","last_clean":"2024-05-14T17:03:37.573215+0000","last_became_active":"2024-05-14T17:01:04.237043+0000","last_became_peered":"2024-05-14T17:01:04.237043+0000","last_unstale":"2024-05-14T17:03:37.573215+0000","last_undegraded":"2024-05-14T17:03:37.573215+0000","last_fullsized":"2024-05-14T17:03:37.573215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:15:33.303302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":240,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.447737+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:03:52.447737+0000","last_peered":"2024-05-14T17:03:52.447737+0000","last_clean":"2024-05-14T17:03:52.447737+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:03:52.447737+0000","last_undegraded":"2024-05-14T17:03:52.447737+0000","last_fullsized":"2024-05-14T17:03:52.447737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674179+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:03:37.674179+0000","last_peered":"2024-05-14T17:03:37.674179+0000","last_clean":"2024-05-14T17:03:37.674179+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:03:37.674179+0000","last_undegraded":"2024-05-14T17:03:37.674179+0000","last_fullsized":"2024-05-14T17:03:37.674179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577611+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:03:37.577611+0000","last_peered":"2024-05-14T17:03:37.577611+0000","last_clean":"2024-05-14T17:03:37.577611+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:03:37.577611+0000","last_undegraded":"2024-05-14T17:03:37.577611+0000","last_fullsized":"2024-05-14T17:03:37.577611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:39.675244+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:03:39.675244+0000","last_peered":"2024-05-14T17:03:39.675244+0000","last_clean":"2024-05-14T17:03:39.675244+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:03:39.675244+0000","last_undegraded":"2024-05-14T17:03:39.675244+0000","last_fullsized":"2024-05-14T17:03:39.675244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687172+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:03:37.687172+0000","last_peered":"2024-05-14T17:03:37.687172+0000","last_clean":"2024-05-14T17:03:37.687172+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:03:37.687172+0000","last_undegraded":"2024-05-14T17:03:37.687172+0000","last_fullsized":"2024-05-14T17:03:37.687172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789348+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:03:37.789348+0000","last_peered":"2024-05-14T17:03:37.789348+0000","last_clean":"2024-05-14T17:03:37.789348+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:03:37.789348+0000","last_undegraded":"2024-05-14T17:03:37.789348+0000","last_fullsized":"2024-05-14T17:03:37.789348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674457+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:03:37.674457+0000","last_peered":"2024-05-14T17:03:37.674457+0000","last_clean":"2024-05-14T17:03:37.674457+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:03:37.674457+0000","last_undegraded":"2024-05-14T17:03:37.674457+0000","last_fullsized":"2024-05-14T17:03:37.674457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688217+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:03:37.688217+0000","last_peered":"2024-05-14T17:03:37.688217+0000","last_clean":"2024-05-14T17:03:37.688217+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:03:37.688217+0000","last_undegraded":"2024-05-14T17:03:37.688217+0000","last_fullsized":"2024-05-14T17:03:37.688217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'100","reported_seq":221,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:53.281414+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:03:53.281414+0000","last_peered":"2024-05-14T17:03:53.281414+0000","last_clean":"2024-05-14T17:03:53.281414+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:03:53.281414+0000","last_undegraded":"2024-05-14T17:03:53.281414+0000","last_fullsized":"2024-05-14T17:03:53.281414+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":184,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674648+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:03:37.674648+0000","last_peered":"2024-05-14T17:03:37.674648+0000","last_clean":"2024-05-14T17:03:37.674648+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:03:37.674648+0000","last_undegraded":"2024-05-14T17:03:37.674648+0000","last_fullsized":"2024-05-14T17:03:37.674648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577683+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:03:37.577683+0000","last_peered":"2024-05-14T17:03:37.577683+0000","last_clean":"2024-05-14T17:03:37.577683+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:03:37.577683+0000","last_undegraded":"2024-05-14T17:03:37.577683+0000","last_fullsized":"2024-05-14T17:03:37.577683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687669+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:03:37.687669+0000","last_peered":"2024-05-14T17:03:37.687669+0000","last_clean":"2024-05-14T17:03:37.687669+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:03:37.687669+0000","last_undegraded":"2024-05-14T17:03:37.687669+0000","last_fullsized":"2024-05-14T17:03:37.687669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686711+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:03:37.686711+0000","last_peered":"2024-05-14T17:03:37.686711+0000","last_clean":"2024-05-14T17:03:37.686711+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:03:37.686711+0000","last_undegraded":"2024-05-14T17:03:37.686711+0000","last_fullsized":"2024-05-14T17:03:37.686711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674192+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:03:37.674192+0000","last_peered":"2024-05-14T17:03:37.674192+0000","last_clean":"2024-05-14T17:03:37.674192+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:03:37.674192+0000","last_undegraded":"2024-05-14T17:03:37.674192+0000","last_fullsized":"2024-05-14T17:03:37.674192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674032+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:03:37.674032+0000","last_peered":"2024-05-14T17:03:37.674032+0000","last_clean":"2024-05-14T17:03:37.674032+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:03:37.674032+0000","last_undegraded":"2024-05-14T17:03:37.674032+0000","last_fullsized":"2024-05-14T17:03:37.674032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686985+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:03:37.686985+0000","last_peered":"2024-05-14T17:03:37.686985+0000","last_clean":"2024-05-14T17:03:37.686985+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:03:37.686985+0000","last_undegraded":"2024-05-14T17:03:37.686985+0000","last_fullsized":"2024-05-14T17:03:37.686985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575638+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:03:37.575638+0000","last_peered":"2024-05-14T17:03:37.575638+0000","last_clean":"2024-05-14T17:03:37.575638+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:03:37.575638+0000","last_undegraded":"2024-05-14T17:03:37.575638+0000","last_fullsized":"2024-05-14T17:03:37.575638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674360+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:03:37.674360+0000","last_peered":"2024-05-14T17:03:37.674360+0000","last_clean":"2024-05-14T17:03:37.674360+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:03:37.674360+0000","last_undegraded":"2024-05-14T17:03:37.674360+0000","last_fullsized":"2024-05-14T17:03:37.674360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687026+0000","last_change":"2024-05-14T17:01:04.243512+0000","last_active":"2024-05-14T17:03:37.687026+0000","last_peered":"2024-05-14T17:03:37.687026+0000","last_clean":"2024-05-14T17:03:37.687026+0000","last_became_active":"2024-05-14T17:01:04.237118+0000","last_became_peered":"2024-05-14T17:01:04.237118+0000","last_unstale":"2024-05-14T17:03:37.687026+0000","last_undegraded":"2024-05-14T17:03:37.687026+0000","last_fullsized":"2024-05-14T17:03:37.687026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:19:16.524481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789265+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:03:37.789265+0000","last_peered":"2024-05-14T17:03:37.789265+0000","last_clean":"2024-05-14T17:03:37.789265+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:03:37.789265+0000","last_undegraded":"2024-05-14T17:03:37.789265+0000","last_fullsized":"2024-05-14T17:03:37.789265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575766+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:03:37.575766+0000","last_peered":"2024-05-14T17:03:37.575766+0000","last_clean":"2024-05-14T17:03:37.575766+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:03:37.575766+0000","last_undegraded":"2024-05-14T17:03:37.575766+0000","last_fullsized":"2024-05-14T17:03:37.575766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789479+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:03:37.789479+0000","last_peered":"2024-05-14T17:03:37.789479+0000","last_clean":"2024-05-14T17:03:37.789479+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:03:37.789479+0000","last_undegraded":"2024-05-14T17:03:37.789479+0000","last_fullsized":"2024-05-14T17:03:37.789479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'65","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.433348+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:03:52.433348+0000","last_peered":"2024-05-14T17:03:52.433348+0000","last_clean":"2024-05-14T17:03:52.433348+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:03:52.433348+0000","last_undegraded":"2024-05-14T17:03:52.433348+0000","last_fullsized":"2024-05-14T17:03:52.433348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789250+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:03:37.789250+0000","last_peered":"2024-05-14T17:03:37.789250+0000","last_clean":"2024-05-14T17:03:37.789250+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:03:37.789250+0000","last_undegraded":"2024-05-14T17:03:37.789250+0000","last_fullsized":"2024-05-14T17:03:37.789250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688103+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:03:37.688103+0000","last_peered":"2024-05-14T17:03:37.688103+0000","last_clean":"2024-05-14T17:03:37.688103+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:03:37.688103+0000","last_undegraded":"2024-05-14T17:03:37.688103+0000","last_fullsized":"2024-05-14T17:03:37.688103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789479+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:03:37.789479+0000","last_peered":"2024-05-14T17:03:37.789479+0000","last_clean":"2024-05-14T17:03:37.789479+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:03:37.789479+0000","last_undegraded":"2024-05-14T17:03:37.789479+0000","last_fullsized":"2024-05-14T17:03:37.789479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687096+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:03:37.687096+0000","last_peered":"2024-05-14T17:03:37.687096+0000","last_clean":"2024-05-14T17:03:37.687096+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:03:37.687096+0000","last_undegraded":"2024-05-14T17:03:37.687096+0000","last_fullsized":"2024-05-14T17:03:37.687096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575519+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:03:37.575519+0000","last_peered":"2024-05-14T17:03:37.575519+0000","last_clean":"2024-05-14T17:03:37.575519+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:03:37.575519+0000","last_undegraded":"2024-05-14T17:03:37.575519+0000","last_fullsized":"2024-05-14T17:03:37.575519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575519+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:03:37.575519+0000","last_peered":"2024-05-14T17:03:37.575519+0000","last_clean":"2024-05-14T17:03:37.575519+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:03:37.575519+0000","last_undegraded":"2024-05-14T17:03:37.575519+0000","last_fullsized":"2024-05-14T17:03:37.575519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687177+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:03:37.687177+0000","last_peered":"2024-05-14T17:03:37.687177+0000","last_clean":"2024-05-14T17:03:37.687177+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:03:37.687177+0000","last_undegraded":"2024-05-14T17:03:37.687177+0000","last_fullsized":"2024-05-14T17:03:37.687177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688376+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:03:37.688376+0000","last_peered":"2024-05-14T17:03:37.688376+0000","last_clean":"2024-05-14T17:03:37.688376+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:03:37.688376+0000","last_undegraded":"2024-05-14T17:03:37.688376+0000","last_fullsized":"2024-05-14T17:03:37.688376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687233+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:03:37.687233+0000","last_peered":"2024-05-14T17:03:37.687233+0000","last_clean":"2024-05-14T17:03:37.687233+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:03:37.687233+0000","last_undegraded":"2024-05-14T17:03:37.687233+0000","last_fullsized":"2024-05-14T17:03:37.687233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575880+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:03:37.575880+0000","last_peered":"2024-05-14T17:03:37.575880+0000","last_clean":"2024-05-14T17:03:37.575880+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:03:37.575880+0000","last_undegraded":"2024-05-14T17:03:37.575880+0000","last_fullsized":"2024-05-14T17:03:37.575880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674537+0000","last_change":"2024-05-14T17:01:04.238941+0000","last_active":"2024-05-14T17:03:37.674537+0000","last_peered":"2024-05-14T17:03:37.674537+0000","last_clean":"2024-05-14T17:03:37.674537+0000","last_became_active":"2024-05-14T17:01:04.238526+0000","last_became_peered":"2024-05-14T17:01:04.238526+0000","last_unstale":"2024-05-14T17:03:37.674537+0000","last_undegraded":"2024-05-14T17:03:37.674537+0000","last_fullsized":"2024-05-14T17:03:37.674537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:41:40.081665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680279+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:03:37.680279+0000","last_peered":"2024-05-14T17:03:37.680279+0000","last_clean":"2024-05-14T17:03:37.680279+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:03:37.680279+0000","last_undegraded":"2024-05-14T17:03:37.680279+0000","last_fullsized":"2024-05-14T17:03:37.680279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.673958+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:03:37.673958+0000","last_peered":"2024-05-14T17:03:37.673958+0000","last_clean":"2024-05-14T17:03:37.673958+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:03:37.673958+0000","last_undegraded":"2024-05-14T17:03:37.673958+0000","last_fullsized":"2024-05-14T17:03:37.673958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686877+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:03:37.686877+0000","last_peered":"2024-05-14T17:03:37.686877+0000","last_clean":"2024-05-14T17:03:37.686877+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:03:37.686877+0000","last_undegraded":"2024-05-14T17:03:37.686877+0000","last_fullsized":"2024-05-14T17:03:37.686877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688787+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:03:37.688787+0000","last_peered":"2024-05-14T17:03:37.688787+0000","last_clean":"2024-05-14T17:03:37.688787+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:03:37.688787+0000","last_undegraded":"2024-05-14T17:03:37.688787+0000","last_fullsized":"2024-05-14T17:03:37.688787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687085+0000","last_change":"2024-05-14T17:01:04.239386+0000","last_active":"2024-05-14T17:03:37.687085+0000","last_peered":"2024-05-14T17:03:37.687085+0000","last_clean":"2024-05-14T17:03:37.687085+0000","last_became_active":"2024-05-14T17:01:04.239053+0000","last_became_peered":"2024-05-14T17:01:04.239053+0000","last_unstale":"2024-05-14T17:03:37.687085+0000","last_undegraded":"2024-05-14T17:03:37.687085+0000","last_fullsized":"2024-05-14T17:03:37.687085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:49.304566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793822,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1612,"num_read_kb":1019,"num_write":417,"num_write_kb":2351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1047,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2381466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645373,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":441660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841601,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29028,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716124,"statfs":{"total":95995035648,"available":95965310976,"internally_reserved":0,"allocated":1937408,"data_stored":1428789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1310720,"data_stored":835078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201356,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36616,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708536,"statfs":{"total":95995035648,"available":95957540864,"internally_reserved":0,"allocated":1310720,"data_stored":834998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1035026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561109,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":905216,"data_stored":441251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":508103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":442522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":508797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149796,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28744,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716408,"statfs":{"total":95995035648,"available":95965601792,"internally_reserved":0,"allocated":1646592,"data_stored":1163010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757339,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37268,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707884,"statfs":{"total":95995035648,"available":95956873216,"internally_reserved":0,"allocated":1855488,"data_stored":1362551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:03:56.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:03:57.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:56 smithi150 ceph-mon[36204]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-14T17:03:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:56 smithi150 ceph-mon[36204]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:03:57.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:56 smithi099 ceph-mon[31851]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-14T17:03:57.044 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:56 smithi099 ceph-mon[31851]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:03:57.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:56 smithi193 ceph-mon[35962]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-14T17:03:57.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:56 smithi193 ceph-mon[35962]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:03:57.301 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:58.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 -- 172.21.15.99:0/3821015602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e006c1d0 msgr2=0x7f00e006c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:58.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/3821015602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e006c1d0 0x7f00e006c650 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009990 tx=0x7f00d4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 -- 172.21.15.99:0/3821015602 shutdown_connections 2024-05-14T17:03:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/3821015602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e0105f70 0x7f00e0106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/3821015602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e0108460 0x7f00e006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/3821015602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e006c1d0 0x7f00e006c650 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 -- 172.21.15.99:0/3821015602 >> 172.21.15.99:0/3821015602 conn(0x7f00e0079410 msgr2=0x7f00e0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:58.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 -- 172.21.15.99:0/3821015602 shutdown_connections 2024-05-14T17:03:59.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.870+0000 7f00e4dbf700 1 -- 172.21.15.99:0/3821015602 wait complete. 2024-05-14T17:03:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 Processor -- start 2024-05-14T17:03:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 -- start start 2024-05-14T17:03:59.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e006c1d0 0x7f00e01a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 0x7f00e01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e0108460 0x7f00e01a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0119d90 con 0x7f00e0105f70 2024-05-14T17:03:59.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0119c10 con 0x7f00e0108460 2024-05-14T17:03:59.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00e4dbf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0119f10 con 0x7f00e006c1d0 2024-05-14T17:03:59.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 0x7f00e01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00ddd9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 0x7f00e01a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44638/0 (socket says 172.21.15.99:44638) 2024-05-14T17:03:59.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00ddd9b700 1 -- 172.21.15.99:0/1562034543 learned_addr learned my addr 172.21.15.99:0/1562034543 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:03:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00de59c700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e006c1d0 0x7f00e01a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.871+0000 7f00ded9d700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e0108460 0x7f00e01a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00ddd9b700 1 -- 172.21.15.99:0/1562034543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e006c1d0 msgr2=0x7f00e01a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:59.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00ddd9b700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e006c1d0 0x7f00e01a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00ddd9b700 1 -- 172.21.15.99:0/1562034543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e0108460 msgr2=0x7f00e01a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:59.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00ddd9b700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e0108460 0x7f00e01a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00ddd9b700 1 -- 172.21.15.99:0/1562034543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e01a52b0 con 0x7f00e0105f70 2024-05-14T17:03:59.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00ddd9b700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 0x7f00e01a07e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f00d4005910 tx=0x7f00d40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:59.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4027070 con 0x7f00e0105f70 2024-05-14T17:03:59.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d4003f20 con 0x7f00e0105f70 2024-05-14T17:03:59.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01a5540 con 0x7f00e0105f70 2024-05-14T17:03:59.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4022c70 con 0x7f00e0105f70 2024-05-14T17:03:59.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.872+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01ace20 con 0x7f00e0105f70 2024-05-14T17:03:59.273 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:58 smithi193 ceph-mon[35962]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-14T17:03:59.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:58 smithi150 ceph-mon[36204]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-14T17:03:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.873+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e006a230 con 0x7f00e0105f70 2024-05-14T17:03:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.874+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f00d402a070 con 0x7f00e0105f70 2024-05-14T17:03:59.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.875+0000 7f00cf7fe700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00c80b2b00 0x7f00c80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.875+0000 7f00de59c700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00c80b2b00 0x7f00c80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.875+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f00d40e3c80 con 0x7f00e0105f70 2024-05-14T17:03:59.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.875+0000 7f00de59c700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00c80b2b00 0x7f00c80b4fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009a10 tx=0x7f00d0009190 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:59.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:58.879+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00d4031070 con 0x7f00e0105f70 2024-05-14T17:03:59.471 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:58 smithi099 ceph-mon[31851]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-14T17:03:59.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.081+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00e0105660 con 0x7f00c80b2b00 2024-05-14T17:03:59.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.088+0000 7f00cf7fe700 1 -- 172.21.15.99:0/1562034543 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257762 (secure 0 0 0) 0x7f00e0105660 con 0x7f00c80b2b00 2024-05-14T17:03:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:03:59.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00c80b2b00 msgr2=0x7f00c80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:59.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00c80b2b00 0x7f00c80b4fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009a10 tx=0x7f00d0009190 comp rx=0 tx=0).stop 2024-05-14T17:03:59.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 msgr2=0x7f00e01a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:59.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 0x7f00e01a07e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f00d4005910 tx=0x7f00d40047b0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 shutdown_connections 2024-05-14T17:03:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00c80b2b00 0x7f00c80b4fc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00e0108460 0x7f00e01a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e006c1d0 0x7f00e01a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 --2- 172.21.15.99:0/1562034543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00e0105f70 0x7f00e01a07e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 >> 172.21.15.99:0/1562034543 conn(0x7f00e0079410 msgr2=0x7f00e0104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:59.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 shutdown_connections 2024-05-14T17:03:59.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:03:59.092+0000 7f00e4dbf700 1 -- 172.21.15.99:0/1562034543 wait complete. 2024-05-14T17:03:59.740 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:03:59.752 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-14T17:03:57.593599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384424,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1756,"num_read_kb":1267,"num_write":554,"num_write_kb":4503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2952887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":907,"ondisk_log_size":907,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372684,"kb_used_data":14832,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569140,"statfs":{"total":1151940427776,"available":1151558799360,"internally_reserved":0,"allocated":15187968,"data_stored":9445800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":260,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002835"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687318+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:03:37.687318+0000","last_peered":"2024-05-14T17:03:37.687318+0000","last_clean":"2024-05-14T17:03:37.687318+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:03:37.687318+0000","last_undegraded":"2024-05-14T17:03:37.687318+0000","last_fullsized":"2024-05-14T17:03:37.687318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687622+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:03:37.687622+0000","last_peered":"2024-05-14T17:03:37.687622+0000","last_clean":"2024-05-14T17:03:37.687622+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:03:37.687622+0000","last_undegraded":"2024-05-14T17:03:37.687622+0000","last_fullsized":"2024-05-14T17:03:37.687622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687406+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:03:37.687406+0000","last_peered":"2024-05-14T17:03:37.687406+0000","last_clean":"2024-05-14T17:03:37.687406+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:03:37.687406+0000","last_undegraded":"2024-05-14T17:03:37.687406+0000","last_fullsized":"2024-05-14T17:03:37.687406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577663+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:03:37.577663+0000","last_peered":"2024-05-14T17:03:37.577663+0000","last_clean":"2024-05-14T17:03:37.577663+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:03:37.577663+0000","last_undegraded":"2024-05-14T17:03:37.577663+0000","last_fullsized":"2024-05-14T17:03:37.577663+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674251+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:03:37.674251+0000","last_peered":"2024-05-14T17:03:37.674251+0000","last_clean":"2024-05-14T17:03:37.674251+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:03:37.674251+0000","last_undegraded":"2024-05-14T17:03:37.674251+0000","last_fullsized":"2024-05-14T17:03:37.674251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683335+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:03:37.683335+0000","last_peered":"2024-05-14T17:03:37.683335+0000","last_clean":"2024-05-14T17:03:37.683335+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:03:37.683335+0000","last_undegraded":"2024-05-14T17:03:37.683335+0000","last_fullsized":"2024-05-14T17:03:37.683335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789151+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:03:37.789151+0000","last_peered":"2024-05-14T17:03:37.789151+0000","last_clean":"2024-05-14T17:03:37.789151+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:03:37.789151+0000","last_undegraded":"2024-05-14T17:03:37.789151+0000","last_fullsized":"2024-05-14T17:03:37.789151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.673941+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:03:37.673941+0000","last_peered":"2024-05-14T17:03:37.673941+0000","last_clean":"2024-05-14T17:03:37.673941+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:03:37.673941+0000","last_undegraded":"2024-05-14T17:03:37.673941+0000","last_fullsized":"2024-05-14T17:03:37.673941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687162+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:03:37.687162+0000","last_peered":"2024-05-14T17:03:37.687162+0000","last_clean":"2024-05-14T17:03:37.687162+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:03:37.687162+0000","last_undegraded":"2024-05-14T17:03:37.687162+0000","last_fullsized":"2024-05-14T17:03:37.687162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683580+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:03:37.683580+0000","last_peered":"2024-05-14T17:03:37.683580+0000","last_clean":"2024-05-14T17:03:37.683580+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:03:37.683580+0000","last_undegraded":"2024-05-14T17:03:37.683580+0000","last_fullsized":"2024-05-14T17:03:37.683580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789385+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:03:37.789385+0000","last_peered":"2024-05-14T17:03:37.789385+0000","last_clean":"2024-05-14T17:03:37.789385+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:03:37.789385+0000","last_undegraded":"2024-05-14T17:03:37.789385+0000","last_fullsized":"2024-05-14T17:03:37.789385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577660+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:03:37.577660+0000","last_peered":"2024-05-14T17:03:37.577660+0000","last_clean":"2024-05-14T17:03:37.577660+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:03:37.577660+0000","last_undegraded":"2024-05-14T17:03:37.577660+0000","last_fullsized":"2024-05-14T17:03:37.577660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688782+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:03:37.688782+0000","last_peered":"2024-05-14T17:03:37.688782+0000","last_clean":"2024-05-14T17:03:37.688782+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:03:37.688782+0000","last_undegraded":"2024-05-14T17:03:37.688782+0000","last_fullsized":"2024-05-14T17:03:37.688782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686967+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:03:37.686967+0000","last_peered":"2024-05-14T17:03:37.686967+0000","last_clean":"2024-05-14T17:03:37.686967+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:03:37.686967+0000","last_undegraded":"2024-05-14T17:03:37.686967+0000","last_fullsized":"2024-05-14T17:03:37.686967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686877+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:03:37.686877+0000","last_peered":"2024-05-14T17:03:37.686877+0000","last_clean":"2024-05-14T17:03:37.686877+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:03:37.686877+0000","last_undegraded":"2024-05-14T17:03:37.686877+0000","last_fullsized":"2024-05-14T17:03:37.686877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686703+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:03:37.686703+0000","last_peered":"2024-05-14T17:03:37.686703+0000","last_clean":"2024-05-14T17:03:37.686703+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:03:37.686703+0000","last_undegraded":"2024-05-14T17:03:37.686703+0000","last_fullsized":"2024-05-14T17:03:37.686703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687437+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:03:37.687437+0000","last_peered":"2024-05-14T17:03:37.687437+0000","last_clean":"2024-05-14T17:03:37.687437+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:03:37.687437+0000","last_undegraded":"2024-05-14T17:03:37.687437+0000","last_fullsized":"2024-05-14T17:03:37.687437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573578+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:03:37.573578+0000","last_peered":"2024-05-14T17:03:37.573578+0000","last_clean":"2024-05-14T17:03:37.573578+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:03:37.573578+0000","last_undegraded":"2024-05-14T17:03:37.573578+0000","last_fullsized":"2024-05-14T17:03:37.573578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575796+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:03:37.575796+0000","last_peered":"2024-05-14T17:03:37.575796+0000","last_clean":"2024-05-14T17:03:37.575796+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:03:37.575796+0000","last_undegraded":"2024-05-14T17:03:37.575796+0000","last_fullsized":"2024-05-14T17:03:37.575796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573421+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:03:37.573421+0000","last_peered":"2024-05-14T17:03:37.573421+0000","last_clean":"2024-05-14T17:03:37.573421+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:03:37.573421+0000","last_undegraded":"2024-05-14T17:03:37.573421+0000","last_fullsized":"2024-05-14T17:03:37.573421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674758+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:03:37.674758+0000","last_peered":"2024-05-14T17:03:37.674758+0000","last_clean":"2024-05-14T17:03:37.674758+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:03:37.674758+0000","last_undegraded":"2024-05-14T17:03:37.674758+0000","last_fullsized":"2024-05-14T17:03:37.674758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674912+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:03:37.674912+0000","last_peered":"2024-05-14T17:03:37.674912+0000","last_clean":"2024-05-14T17:03:37.674912+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:03:37.674912+0000","last_undegraded":"2024-05-14T17:03:37.674912+0000","last_fullsized":"2024-05-14T17:03:37.674912+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687882+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:03:37.687882+0000","last_peered":"2024-05-14T17:03:37.687882+0000","last_clean":"2024-05-14T17:03:37.687882+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:03:37.687882+0000","last_undegraded":"2024-05-14T17:03:37.687882+0000","last_fullsized":"2024-05-14T17:03:37.687882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674597+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:03:37.674597+0000","last_peered":"2024-05-14T17:03:37.674597+0000","last_clean":"2024-05-14T17:03:37.674597+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:03:37.674597+0000","last_undegraded":"2024-05-14T17:03:37.674597+0000","last_fullsized":"2024-05-14T17:03:37.674597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686859+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:03:37.686859+0000","last_peered":"2024-05-14T17:03:37.686859+0000","last_clean":"2024-05-14T17:03:37.686859+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:03:37.686859+0000","last_undegraded":"2024-05-14T17:03:37.686859+0000","last_fullsized":"2024-05-14T17:03:37.686859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674465+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:03:37.674465+0000","last_peered":"2024-05-14T17:03:37.674465+0000","last_clean":"2024-05-14T17:03:37.674465+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:03:37.674465+0000","last_undegraded":"2024-05-14T17:03:37.674465+0000","last_fullsized":"2024-05-14T17:03:37.674465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680651+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:03:37.680651+0000","last_peered":"2024-05-14T17:03:37.680651+0000","last_clean":"2024-05-14T17:03:37.680651+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:03:37.680651+0000","last_undegraded":"2024-05-14T17:03:37.680651+0000","last_fullsized":"2024-05-14T17:03:37.680651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":261,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.541061+0000","last_change":"2024-05-14T17:01:04.271215+0000","last_active":"2024-05-14T17:03:52.541061+0000","last_peered":"2024-05-14T17:03:52.541061+0000","last_clean":"2024-05-14T17:03:52.541061+0000","last_became_active":"2024-05-14T17:01:04.267186+0000","last_became_peered":"2024-05-14T17:01:04.267186+0000","last_unstale":"2024-05-14T17:03:52.541061+0000","last_undegraded":"2024-05-14T17:03:52.541061+0000","last_fullsized":"2024-05-14T17:03:52.541061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:01.465730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674220+0000","last_change":"2024-05-14T17:01:04.241079+0000","last_active":"2024-05-14T17:03:37.674220+0000","last_peered":"2024-05-14T17:03:37.674220+0000","last_clean":"2024-05-14T17:03:37.674220+0000","last_became_active":"2024-05-14T17:01:04.240698+0000","last_became_peered":"2024-05-14T17:01:04.240698+0000","last_unstale":"2024-05-14T17:03:37.674220+0000","last_undegraded":"2024-05-14T17:03:37.674220+0000","last_fullsized":"2024-05-14T17:03:37.674220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:10.589991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683244+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:03:37.683244+0000","last_peered":"2024-05-14T17:03:37.683244+0000","last_clean":"2024-05-14T17:03:37.683244+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:03:37.683244+0000","last_undegraded":"2024-05-14T17:03:37.683244+0000","last_fullsized":"2024-05-14T17:03:37.683244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680431+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:03:37.680431+0000","last_peered":"2024-05-14T17:03:37.680431+0000","last_clean":"2024-05-14T17:03:37.680431+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:03:37.680431+0000","last_undegraded":"2024-05-14T17:03:37.680431+0000","last_fullsized":"2024-05-14T17:03:37.680431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687440+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:03:37.687440+0000","last_peered":"2024-05-14T17:03:37.687440+0000","last_clean":"2024-05-14T17:03:37.687440+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:03:37.687440+0000","last_undegraded":"2024-05-14T17:03:37.687440+0000","last_fullsized":"2024-05-14T17:03:37.687440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573316+0000","last_change":"2024-05-14T17:01:04.236554+0000","last_active":"2024-05-14T17:03:37.573316+0000","last_peered":"2024-05-14T17:03:37.573316+0000","last_clean":"2024-05-14T17:03:37.573316+0000","last_became_active":"2024-05-14T17:01:04.236203+0000","last_became_peered":"2024-05-14T17:01:04.236203+0000","last_unstale":"2024-05-14T17:03:37.573316+0000","last_undegraded":"2024-05-14T17:03:37.573316+0000","last_fullsized":"2024-05-14T17:03:37.573316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:34.421246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680247+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:03:37.680247+0000","last_peered":"2024-05-14T17:03:37.680247+0000","last_clean":"2024-05-14T17:03:37.680247+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:03:37.680247+0000","last_undegraded":"2024-05-14T17:03:37.680247+0000","last_fullsized":"2024-05-14T17:03:37.680247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573284+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:03:37.573284+0000","last_peered":"2024-05-14T17:03:37.573284+0000","last_clean":"2024-05-14T17:03:37.573284+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:03:37.573284+0000","last_undegraded":"2024-05-14T17:03:37.573284+0000","last_fullsized":"2024-05-14T17:03:37.573284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":240,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.540607+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:03:52.540607+0000","last_peered":"2024-05-14T17:03:52.540607+0000","last_clean":"2024-05-14T17:03:52.540607+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:03:52.540607+0000","last_undegraded":"2024-05-14T17:03:52.540607+0000","last_fullsized":"2024-05-14T17:03:52.540607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687113+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:03:37.687113+0000","last_peered":"2024-05-14T17:03:37.687113+0000","last_clean":"2024-05-14T17:03:37.687113+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:03:37.687113+0000","last_undegraded":"2024-05-14T17:03:37.687113+0000","last_fullsized":"2024-05-14T17:03:37.687113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789349+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:03:37.789349+0000","last_peered":"2024-05-14T17:03:37.789349+0000","last_clean":"2024-05-14T17:03:37.789349+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:03:37.789349+0000","last_undegraded":"2024-05-14T17:03:37.789349+0000","last_fullsized":"2024-05-14T17:03:37.789349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687808+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:03:37.687808+0000","last_peered":"2024-05-14T17:03:37.687808+0000","last_clean":"2024-05-14T17:03:37.687808+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:03:37.687808+0000","last_undegraded":"2024-05-14T17:03:37.687808+0000","last_fullsized":"2024-05-14T17:03:37.687808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680791+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:03:37.680791+0000","last_peered":"2024-05-14T17:03:37.680791+0000","last_clean":"2024-05-14T17:03:37.680791+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:03:37.680791+0000","last_undegraded":"2024-05-14T17:03:37.680791+0000","last_fullsized":"2024-05-14T17:03:37.680791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686928+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:03:37.686928+0000","last_peered":"2024-05-14T17:03:37.686928+0000","last_clean":"2024-05-14T17:03:37.686928+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:03:37.686928+0000","last_undegraded":"2024-05-14T17:03:37.686928+0000","last_fullsized":"2024-05-14T17:03:37.686928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":488,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:53.541533+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:03:53.541533+0000","last_peered":"2024-05-14T17:03:53.541533+0000","last_clean":"2024-05-14T17:03:53.541533+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:03:53.541533+0000","last_undegraded":"2024-05-14T17:03:53.541533+0000","last_fullsized":"2024-05-14T17:03:53.541533+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":424,"num_read_kb":278,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674413+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:03:37.674413+0000","last_peered":"2024-05-14T17:03:37.674413+0000","last_clean":"2024-05-14T17:03:37.674413+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:03:37.674413+0000","last_undegraded":"2024-05-14T17:03:37.674413+0000","last_fullsized":"2024-05-14T17:03:37.674413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687117+0000","last_change":"2024-05-14T17:01:04.249537+0000","last_active":"2024-05-14T17:03:37.687117+0000","last_peered":"2024-05-14T17:03:37.687117+0000","last_clean":"2024-05-14T17:03:37.687117+0000","last_became_active":"2024-05-14T17:01:04.239963+0000","last_became_peered":"2024-05-14T17:01:04.239963+0000","last_unstale":"2024-05-14T17:03:37.687117+0000","last_undegraded":"2024-05-14T17:03:37.687117+0000","last_fullsized":"2024-05-14T17:03:37.687117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:00:51.798116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688287+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:03:37.688287+0000","last_peered":"2024-05-14T17:03:37.688287+0000","last_clean":"2024-05-14T17:03:37.688287+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:03:37.688287+0000","last_undegraded":"2024-05-14T17:03:37.688287+0000","last_fullsized":"2024-05-14T17:03:37.688287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680371+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:03:37.680371+0000","last_peered":"2024-05-14T17:03:37.680371+0000","last_clean":"2024-05-14T17:03:37.680371+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:03:37.680371+0000","last_undegraded":"2024-05-14T17:03:37.680371+0000","last_fullsized":"2024-05-14T17:03:37.680371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789348+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:03:37.789348+0000","last_peered":"2024-05-14T17:03:37.789348+0000","last_clean":"2024-05-14T17:03:37.789348+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:03:37.789348+0000","last_undegraded":"2024-05-14T17:03:37.789348+0000","last_fullsized":"2024-05-14T17:03:37.789348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.448350+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:03:52.448350+0000","last_peered":"2024-05-14T17:03:52.448350+0000","last_clean":"2024-05-14T17:03:52.448350+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:03:52.448350+0000","last_undegraded":"2024-05-14T17:03:52.448350+0000","last_fullsized":"2024-05-14T17:03:52.448350+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789088+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:03:37.789088+0000","last_peered":"2024-05-14T17:03:37.789088+0000","last_clean":"2024-05-14T17:03:37.789088+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:03:37.789088+0000","last_undegraded":"2024-05-14T17:03:37.789088+0000","last_fullsized":"2024-05-14T17:03:37.789088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674198+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:03:37.674198+0000","last_peered":"2024-05-14T17:03:37.674198+0000","last_clean":"2024-05-14T17:03:37.674198+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:03:37.674198+0000","last_undegraded":"2024-05-14T17:03:37.674198+0000","last_fullsized":"2024-05-14T17:03:37.674198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680298+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:03:37.680298+0000","last_peered":"2024-05-14T17:03:37.680298+0000","last_clean":"2024-05-14T17:03:37.680298+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:03:37.680298+0000","last_undegraded":"2024-05-14T17:03:37.680298+0000","last_fullsized":"2024-05-14T17:03:37.680298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674493+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:03:37.674493+0000","last_peered":"2024-05-14T17:03:37.674493+0000","last_clean":"2024-05-14T17:03:37.674493+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:03:37.674493+0000","last_undegraded":"2024-05-14T17:03:37.674493+0000","last_fullsized":"2024-05-14T17:03:37.674493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687257+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:03:37.687257+0000","last_peered":"2024-05-14T17:03:37.687257+0000","last_clean":"2024-05-14T17:03:37.687257+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:03:37.687257+0000","last_undegraded":"2024-05-14T17:03:37.687257+0000","last_fullsized":"2024-05-14T17:03:37.687257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573679+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:03:37.573679+0000","last_peered":"2024-05-14T17:03:37.573679+0000","last_clean":"2024-05-14T17:03:37.573679+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:03:37.573679+0000","last_undegraded":"2024-05-14T17:03:37.573679+0000","last_fullsized":"2024-05-14T17:03:37.573679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683414+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:03:37.683414+0000","last_peered":"2024-05-14T17:03:37.683414+0000","last_clean":"2024-05-14T17:03:37.683414+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:03:37.683414+0000","last_undegraded":"2024-05-14T17:03:37.683414+0000","last_fullsized":"2024-05-14T17:03:37.683414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789470+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:03:37.789470+0000","last_peered":"2024-05-14T17:03:37.789470+0000","last_clean":"2024-05-14T17:03:37.789470+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:03:37.789470+0000","last_undegraded":"2024-05-14T17:03:37.789470+0000","last_fullsized":"2024-05-14T17:03:37.789470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683335+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:03:37.683335+0000","last_peered":"2024-05-14T17:03:37.683335+0000","last_clean":"2024-05-14T17:03:37.683335+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:03:37.683335+0000","last_undegraded":"2024-05-14T17:03:37.683335+0000","last_fullsized":"2024-05-14T17:03:37.683335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573215+0000","last_change":"2024-05-14T17:01:04.237414+0000","last_active":"2024-05-14T17:03:37.573215+0000","last_peered":"2024-05-14T17:03:37.573215+0000","last_clean":"2024-05-14T17:03:37.573215+0000","last_became_active":"2024-05-14T17:01:04.237043+0000","last_became_peered":"2024-05-14T17:01:04.237043+0000","last_unstale":"2024-05-14T17:03:37.573215+0000","last_undegraded":"2024-05-14T17:03:37.573215+0000","last_fullsized":"2024-05-14T17:03:37.573215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:15:33.303302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":240,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.447737+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:03:52.447737+0000","last_peered":"2024-05-14T17:03:52.447737+0000","last_clean":"2024-05-14T17:03:52.447737+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:03:52.447737+0000","last_undegraded":"2024-05-14T17:03:52.447737+0000","last_fullsized":"2024-05-14T17:03:52.447737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674179+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:03:37.674179+0000","last_peered":"2024-05-14T17:03:37.674179+0000","last_clean":"2024-05-14T17:03:37.674179+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:03:37.674179+0000","last_undegraded":"2024-05-14T17:03:37.674179+0000","last_fullsized":"2024-05-14T17:03:37.674179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577611+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:03:37.577611+0000","last_peered":"2024-05-14T17:03:37.577611+0000","last_clean":"2024-05-14T17:03:37.577611+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:03:37.577611+0000","last_undegraded":"2024-05-14T17:03:37.577611+0000","last_fullsized":"2024-05-14T17:03:37.577611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:39.675244+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:03:39.675244+0000","last_peered":"2024-05-14T17:03:39.675244+0000","last_clean":"2024-05-14T17:03:39.675244+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:03:39.675244+0000","last_undegraded":"2024-05-14T17:03:39.675244+0000","last_fullsized":"2024-05-14T17:03:39.675244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687172+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:03:37.687172+0000","last_peered":"2024-05-14T17:03:37.687172+0000","last_clean":"2024-05-14T17:03:37.687172+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:03:37.687172+0000","last_undegraded":"2024-05-14T17:03:37.687172+0000","last_fullsized":"2024-05-14T17:03:37.687172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789348+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:03:37.789348+0000","last_peered":"2024-05-14T17:03:37.789348+0000","last_clean":"2024-05-14T17:03:37.789348+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:03:37.789348+0000","last_undegraded":"2024-05-14T17:03:37.789348+0000","last_fullsized":"2024-05-14T17:03:37.789348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674457+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:03:37.674457+0000","last_peered":"2024-05-14T17:03:37.674457+0000","last_clean":"2024-05-14T17:03:37.674457+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:03:37.674457+0000","last_undegraded":"2024-05-14T17:03:37.674457+0000","last_fullsized":"2024-05-14T17:03:37.674457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688217+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:03:37.688217+0000","last_peered":"2024-05-14T17:03:37.688217+0000","last_clean":"2024-05-14T17:03:37.688217+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:03:37.688217+0000","last_undegraded":"2024-05-14T17:03:37.688217+0000","last_fullsized":"2024-05-14T17:03:37.688217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'100","reported_seq":221,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:53.281414+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:03:53.281414+0000","last_peered":"2024-05-14T17:03:53.281414+0000","last_clean":"2024-05-14T17:03:53.281414+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:03:53.281414+0000","last_undegraded":"2024-05-14T17:03:53.281414+0000","last_fullsized":"2024-05-14T17:03:53.281414+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":184,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674648+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:03:37.674648+0000","last_peered":"2024-05-14T17:03:37.674648+0000","last_clean":"2024-05-14T17:03:37.674648+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:03:37.674648+0000","last_undegraded":"2024-05-14T17:03:37.674648+0000","last_fullsized":"2024-05-14T17:03:37.674648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577683+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:03:37.577683+0000","last_peered":"2024-05-14T17:03:37.577683+0000","last_clean":"2024-05-14T17:03:37.577683+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:03:37.577683+0000","last_undegraded":"2024-05-14T17:03:37.577683+0000","last_fullsized":"2024-05-14T17:03:37.577683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687669+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:03:37.687669+0000","last_peered":"2024-05-14T17:03:37.687669+0000","last_clean":"2024-05-14T17:03:37.687669+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:03:37.687669+0000","last_undegraded":"2024-05-14T17:03:37.687669+0000","last_fullsized":"2024-05-14T17:03:37.687669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686711+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:03:37.686711+0000","last_peered":"2024-05-14T17:03:37.686711+0000","last_clean":"2024-05-14T17:03:37.686711+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:03:37.686711+0000","last_undegraded":"2024-05-14T17:03:37.686711+0000","last_fullsized":"2024-05-14T17:03:37.686711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674192+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:03:37.674192+0000","last_peered":"2024-05-14T17:03:37.674192+0000","last_clean":"2024-05-14T17:03:37.674192+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:03:37.674192+0000","last_undegraded":"2024-05-14T17:03:37.674192+0000","last_fullsized":"2024-05-14T17:03:37.674192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674032+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:03:37.674032+0000","last_peered":"2024-05-14T17:03:37.674032+0000","last_clean":"2024-05-14T17:03:37.674032+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:03:37.674032+0000","last_undegraded":"2024-05-14T17:03:37.674032+0000","last_fullsized":"2024-05-14T17:03:37.674032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686985+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:03:37.686985+0000","last_peered":"2024-05-14T17:03:37.686985+0000","last_clean":"2024-05-14T17:03:37.686985+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:03:37.686985+0000","last_undegraded":"2024-05-14T17:03:37.686985+0000","last_fullsized":"2024-05-14T17:03:37.686985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575638+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:03:37.575638+0000","last_peered":"2024-05-14T17:03:37.575638+0000","last_clean":"2024-05-14T17:03:37.575638+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:03:37.575638+0000","last_undegraded":"2024-05-14T17:03:37.575638+0000","last_fullsized":"2024-05-14T17:03:37.575638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674360+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:03:37.674360+0000","last_peered":"2024-05-14T17:03:37.674360+0000","last_clean":"2024-05-14T17:03:37.674360+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:03:37.674360+0000","last_undegraded":"2024-05-14T17:03:37.674360+0000","last_fullsized":"2024-05-14T17:03:37.674360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687026+0000","last_change":"2024-05-14T17:01:04.243512+0000","last_active":"2024-05-14T17:03:37.687026+0000","last_peered":"2024-05-14T17:03:37.687026+0000","last_clean":"2024-05-14T17:03:37.687026+0000","last_became_active":"2024-05-14T17:01:04.237118+0000","last_became_peered":"2024-05-14T17:01:04.237118+0000","last_unstale":"2024-05-14T17:03:37.687026+0000","last_undegraded":"2024-05-14T17:03:37.687026+0000","last_fullsized":"2024-05-14T17:03:37.687026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:19:16.524481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789265+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:03:37.789265+0000","last_peered":"2024-05-14T17:03:37.789265+0000","last_clean":"2024-05-14T17:03:37.789265+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:03:37.789265+0000","last_undegraded":"2024-05-14T17:03:37.789265+0000","last_fullsized":"2024-05-14T17:03:37.789265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575766+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:03:37.575766+0000","last_peered":"2024-05-14T17:03:37.575766+0000","last_clean":"2024-05-14T17:03:37.575766+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:03:37.575766+0000","last_undegraded":"2024-05-14T17:03:37.575766+0000","last_fullsized":"2024-05-14T17:03:37.575766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789479+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:03:37.789479+0000","last_peered":"2024-05-14T17:03:37.789479+0000","last_clean":"2024-05-14T17:03:37.789479+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:03:37.789479+0000","last_undegraded":"2024-05-14T17:03:37.789479+0000","last_fullsized":"2024-05-14T17:03:37.789479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'65","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.433348+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:03:52.433348+0000","last_peered":"2024-05-14T17:03:52.433348+0000","last_clean":"2024-05-14T17:03:52.433348+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:03:52.433348+0000","last_undegraded":"2024-05-14T17:03:52.433348+0000","last_fullsized":"2024-05-14T17:03:52.433348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789250+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:03:37.789250+0000","last_peered":"2024-05-14T17:03:37.789250+0000","last_clean":"2024-05-14T17:03:37.789250+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:03:37.789250+0000","last_undegraded":"2024-05-14T17:03:37.789250+0000","last_fullsized":"2024-05-14T17:03:37.789250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688103+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:03:37.688103+0000","last_peered":"2024-05-14T17:03:37.688103+0000","last_clean":"2024-05-14T17:03:37.688103+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:03:37.688103+0000","last_undegraded":"2024-05-14T17:03:37.688103+0000","last_fullsized":"2024-05-14T17:03:37.688103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789479+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:03:37.789479+0000","last_peered":"2024-05-14T17:03:37.789479+0000","last_clean":"2024-05-14T17:03:37.789479+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:03:37.789479+0000","last_undegraded":"2024-05-14T17:03:37.789479+0000","last_fullsized":"2024-05-14T17:03:37.789479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687096+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:03:37.687096+0000","last_peered":"2024-05-14T17:03:37.687096+0000","last_clean":"2024-05-14T17:03:37.687096+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:03:37.687096+0000","last_undegraded":"2024-05-14T17:03:37.687096+0000","last_fullsized":"2024-05-14T17:03:37.687096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575519+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:03:37.575519+0000","last_peered":"2024-05-14T17:03:37.575519+0000","last_clean":"2024-05-14T17:03:37.575519+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:03:37.575519+0000","last_undegraded":"2024-05-14T17:03:37.575519+0000","last_fullsized":"2024-05-14T17:03:37.575519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575519+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:03:37.575519+0000","last_peered":"2024-05-14T17:03:37.575519+0000","last_clean":"2024-05-14T17:03:37.575519+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:03:37.575519+0000","last_undegraded":"2024-05-14T17:03:37.575519+0000","last_fullsized":"2024-05-14T17:03:37.575519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687177+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:03:37.687177+0000","last_peered":"2024-05-14T17:03:37.687177+0000","last_clean":"2024-05-14T17:03:37.687177+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:03:37.687177+0000","last_undegraded":"2024-05-14T17:03:37.687177+0000","last_fullsized":"2024-05-14T17:03:37.687177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688376+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:03:37.688376+0000","last_peered":"2024-05-14T17:03:37.688376+0000","last_clean":"2024-05-14T17:03:37.688376+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:03:37.688376+0000","last_undegraded":"2024-05-14T17:03:37.688376+0000","last_fullsized":"2024-05-14T17:03:37.688376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687233+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:03:37.687233+0000","last_peered":"2024-05-14T17:03:37.687233+0000","last_clean":"2024-05-14T17:03:37.687233+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:03:37.687233+0000","last_undegraded":"2024-05-14T17:03:37.687233+0000","last_fullsized":"2024-05-14T17:03:37.687233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575880+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:03:37.575880+0000","last_peered":"2024-05-14T17:03:37.575880+0000","last_clean":"2024-05-14T17:03:37.575880+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:03:37.575880+0000","last_undegraded":"2024-05-14T17:03:37.575880+0000","last_fullsized":"2024-05-14T17:03:37.575880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674537+0000","last_change":"2024-05-14T17:01:04.238941+0000","last_active":"2024-05-14T17:03:37.674537+0000","last_peered":"2024-05-14T17:03:37.674537+0000","last_clean":"2024-05-14T17:03:37.674537+0000","last_became_active":"2024-05-14T17:01:04.238526+0000","last_became_peered":"2024-05-14T17:01:04.238526+0000","last_unstale":"2024-05-14T17:03:37.674537+0000","last_undegraded":"2024-05-14T17:03:37.674537+0000","last_fullsized":"2024-05-14T17:03:37.674537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:41:40.081665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680279+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:03:37.680279+0000","last_peered":"2024-05-14T17:03:37.680279+0000","last_clean":"2024-05-14T17:03:37.680279+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:03:37.680279+0000","last_undegraded":"2024-05-14T17:03:37.680279+0000","last_fullsized":"2024-05-14T17:03:37.680279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.673958+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:03:37.673958+0000","last_peered":"2024-05-14T17:03:37.673958+0000","last_clean":"2024-05-14T17:03:37.673958+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:03:37.673958+0000","last_undegraded":"2024-05-14T17:03:37.673958+0000","last_fullsized":"2024-05-14T17:03:37.673958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686877+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:03:37.686877+0000","last_peered":"2024-05-14T17:03:37.686877+0000","last_clean":"2024-05-14T17:03:37.686877+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:03:37.686877+0000","last_undegraded":"2024-05-14T17:03:37.686877+0000","last_fullsized":"2024-05-14T17:03:37.686877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688787+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:03:37.688787+0000","last_peered":"2024-05-14T17:03:37.688787+0000","last_clean":"2024-05-14T17:03:37.688787+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:03:37.688787+0000","last_undegraded":"2024-05-14T17:03:37.688787+0000","last_fullsized":"2024-05-14T17:03:37.688787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687085+0000","last_change":"2024-05-14T17:01:04.239386+0000","last_active":"2024-05-14T17:03:37.687085+0000","last_peered":"2024-05-14T17:03:37.687085+0000","last_clean":"2024-05-14T17:03:37.687085+0000","last_became_active":"2024-05-14T17:01:04.239053+0000","last_became_peered":"2024-05-14T17:01:04.239053+0000","last_unstale":"2024-05-14T17:03:37.687085+0000","last_undegraded":"2024-05-14T17:03:37.687085+0000","last_fullsized":"2024-05-14T17:03:37.687085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:49.304566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793822,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1612,"num_read_kb":1019,"num_write":417,"num_write_kb":2351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1047,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2381466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645373,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":441660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841601,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29028,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716124,"statfs":{"total":95995035648,"available":95965310976,"internally_reserved":0,"allocated":1937408,"data_stored":1428789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1310720,"data_stored":835078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201356,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36616,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708536,"statfs":{"total":95995035648,"available":95957540864,"internally_reserved":0,"allocated":1310720,"data_stored":834998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1035026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561109,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":905216,"data_stored":441251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":508103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":442522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":508797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149796,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28744,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716408,"statfs":{"total":95995035648,"available":95965601792,"internally_reserved":0,"allocated":1646592,"data_stored":1163010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757339,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37268,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707884,"statfs":{"total":95995035648,"available":95956873216,"internally_reserved":0,"allocated":1855488,"data_stored":1362551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:03:59.794 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T17:03:59.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T17:03:59.840 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:03:59 smithi099 ceph-mon[31851]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:03:59.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:03:59 smithi150 ceph-mon[36204]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:04:00.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:03:59 smithi193 ceph-mon[35962]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:04:00.060 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:00.397 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:04:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:04:00.147+0000 7f8e5390a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:04:00.413 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:04:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:04:00.147+0000 7f8e5390a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:04:00.502 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:04:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:04:00.147+0000 7f8e5390a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:04:00.515 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:04:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:04:00.147+0000 7f8e5390a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:04:00.526 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:04:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x[32082]: 2024-05-14T17:04:00.147+0000 7f8e5390a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T17:04:00.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:00 smithi150 ceph-mon[36204]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-14T17:04:01.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:00 smithi099 ceph-mon[31851]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-14T17:04:01.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:00 smithi193 ceph-mon[35962]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-14T17:04:01.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 -- 172.21.15.99:0/4061160959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 msgr2=0x7f207c11f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 --2- 172.21.15.99:0/4061160959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c11f110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f206c00b0a0 tx=0x7f206c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 -- 172.21.15.99:0/4061160959 shutdown_connections 2024-05-14T17:04:01.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 --2- 172.21.15.99:0/4061160959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c11f110 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 --2- 172.21.15.99:0/4061160959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 0x7f207c10a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 --2- 172.21.15.99:0/4061160959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 0x7f207c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 -- 172.21.15.99:0/4061160959 >> 172.21.15.99:0/4061160959 conn(0x7f207c06e6b0 msgr2=0x7f207c06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:01.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 -- 172.21.15.99:0/4061160959 shutdown_connections 2024-05-14T17:04:01.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.675+0000 7f20827ea700 1 -- 172.21.15.99:0/4061160959 wait complete. 2024-05-14T17:04:01.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.676+0000 7f20827ea700 1 Processor -- start 2024-05-14T17:04:01.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.676+0000 7f20827ea700 1 -- start start 2024-05-14T17:04:01.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f20827ea700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 0x7f207c1ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:01.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f20827ea700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 0x7f207c1aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:01.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f20827ea700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c1a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:01.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f20827ea700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f207c122580 con 0x7f207c11cd20 2024-05-14T17:04:01.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f20827ea700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f207c122400 con 0x7f207c111d20 2024-05-14T17:04:01.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f20827ea700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f207c122700 con 0x7f207c10aa80 2024-05-14T17:04:01.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f2080d87700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c1a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:02.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 0x7f207c1ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:02.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f2080d87700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c1a8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44666/0 (socket says 172.21.15.99:44666) 2024-05-14T17:04:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f2080d87700 1 -- 172.21.15.99:0/1335408147 learned_addr learned my addr 172.21.15.99:0/1335408147 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:02.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207b7fe700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 0x7f207c1aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:02.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207bfff700 1 -- 172.21.15.99:0/1335408147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 msgr2=0x7f207c1aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207bfff700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 0x7f207c1aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207bfff700 1 -- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 msgr2=0x7f207c1a8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:02.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207bfff700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c1a8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207bfff700 1 -- 172.21.15.99:0/1335408147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c1a9020 con 0x7f207c10aa80 2024-05-14T17:04:02.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.677+0000 7f207b7fe700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 0x7f207c1aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:02.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.678+0000 7f207bfff700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 0x7f207c1ae6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f207400e9f0 tx=0x7f207400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:02.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.678+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207400cbd0 con 0x7f207c10aa80 2024-05-14T17:04:02.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.678+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207c1a9310 con 0x7f207c10aa80 2024-05-14T17:04:02.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.678+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207400cd70 con 0x7f207c10aa80 2024-05-14T17:04:02.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.678+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074018680 con 0x7f207c10aa80 2024-05-14T17:04:02.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.678+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207c1b5520 con 0x7f207c10aa80 2024-05-14T17:04:02.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.679+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207c06fc40 con 0x7f207c10aa80 2024-05-14T17:04:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.680+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f207404ab90 con 0x7f207c10aa80 2024-05-14T17:04:02.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.681+0000 7f20797fa700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f20680b2b10 0x7f20680b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:02.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.681+0000 7f207b7fe700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f20680b2b10 0x7f20680b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:02.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.681+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f20740e2a20 con 0x7f207c10aa80 2024-05-14T17:04:02.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.681+0000 7f207b7fe700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f20680b2b10 0x7f20680b4fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f207c111270 tx=0x7f206400b2c0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:02.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.684+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20740aa180 con 0x7f207c10aa80 2024-05-14T17:04:02.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.889+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f207c063980 con 0x7f20680b2b10 2024-05-14T17:04:02.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.896+0000 7f20797fa700 1 -- 172.21.15.99:0/1335408147 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257762 (secure 0 0 0) 0x7f207c063980 con 0x7f20680b2b10 2024-05-14T17:04:02.422 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:04:02.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f20680b2b10 msgr2=0x7f20680b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:02.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f20680b2b10 0x7f20680b4fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f207c111270 tx=0x7f206400b2c0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 msgr2=0x7f207c1ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:02.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 0x7f207c1ae6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f207400e9f0 tx=0x7f207400ed00 comp rx=0 tx=0).stop 2024-05-14T17:04:02.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 shutdown_connections 2024-05-14T17:04:02.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f20680b2b10 0x7f20680b4fd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f207c11cd20 0x7f207c1a8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f207c10aa80 0x7f207c1ae6c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 --2- 172.21.15.99:0/1335408147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f207c111d20 0x7f207c1aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:02.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 >> 172.21.15.99:0/1335408147 conn(0x7f207c06e6b0 msgr2=0x7f207c11aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 shutdown_connections 2024-05-14T17:04:02.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:01.900+0000 7f20827ea700 1 -- 172.21.15.99:0/1335408147 wait complete. 2024-05-14T17:04:02.627 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-05-14T17:04:02.636 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-14T17:04:01.594915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384424,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1786,"num_read_kb":1282,"num_write":554,"num_write_kb":4503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2952887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":907,"ondisk_log_size":907,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372564,"kb_used_data":14832,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569260,"statfs":{"total":1151940427776,"available":1151558922240,"internally_reserved":0,"allocated":15187968,"data_stored":9445800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":260,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003286"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687318+0000","last_change":"2024-05-14T17:00:10.063487+0000","last_active":"2024-05-14T17:03:37.687318+0000","last_peered":"2024-05-14T17:03:37.687318+0000","last_clean":"2024-05-14T17:03:37.687318+0000","last_became_active":"2024-05-14T17:00:10.063029+0000","last_became_peered":"2024-05-14T17:00:10.063029+0000","last_unstale":"2024-05-14T17:03:37.687318+0000","last_undegraded":"2024-05-14T17:03:37.687318+0000","last_fullsized":"2024-05-14T17:03:37.687318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:24.095292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687622+0000","last_change":"2024-05-14T17:00:10.059129+0000","last_active":"2024-05-14T17:03:37.687622+0000","last_peered":"2024-05-14T17:03:37.687622+0000","last_clean":"2024-05-14T17:03:37.687622+0000","last_became_active":"2024-05-14T17:00:10.058803+0000","last_became_peered":"2024-05-14T17:00:10.058803+0000","last_unstale":"2024-05-14T17:03:37.687622+0000","last_undegraded":"2024-05-14T17:03:37.687622+0000","last_fullsized":"2024-05-14T17:03:37.687622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:27:24.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687406+0000","last_change":"2024-05-14T17:00:10.062272+0000","last_active":"2024-05-14T17:03:37.687406+0000","last_peered":"2024-05-14T17:03:37.687406+0000","last_clean":"2024-05-14T17:03:37.687406+0000","last_became_active":"2024-05-14T17:00:10.061737+0000","last_became_peered":"2024-05-14T17:00:10.061737+0000","last_unstale":"2024-05-14T17:03:37.687406+0000","last_undegraded":"2024-05-14T17:03:37.687406+0000","last_fullsized":"2024-05-14T17:03:37.687406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:33.367157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577663+0000","last_change":"2024-05-14T17:00:10.063039+0000","last_active":"2024-05-14T17:03:37.577663+0000","last_peered":"2024-05-14T17:03:37.577663+0000","last_clean":"2024-05-14T17:03:37.577663+0000","last_became_active":"2024-05-14T17:00:10.062374+0000","last_became_peered":"2024-05-14T17:00:10.062374+0000","last_unstale":"2024-05-14T17:03:37.577663+0000","last_undegraded":"2024-05-14T17:03:37.577663+0000","last_fullsized":"2024-05-14T17:03:37.577663+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:45.775442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674251+0000","last_change":"2024-05-14T17:00:10.059358+0000","last_active":"2024-05-14T17:03:37.674251+0000","last_peered":"2024-05-14T17:03:37.674251+0000","last_clean":"2024-05-14T17:03:37.674251+0000","last_became_active":"2024-05-14T17:00:10.058913+0000","last_became_peered":"2024-05-14T17:00:10.058913+0000","last_unstale":"2024-05-14T17:03:37.674251+0000","last_undegraded":"2024-05-14T17:03:37.674251+0000","last_fullsized":"2024-05-14T17:03:37.674251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:08:04.667840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683335+0000","last_change":"2024-05-14T17:00:10.055327+0000","last_active":"2024-05-14T17:03:37.683335+0000","last_peered":"2024-05-14T17:03:37.683335+0000","last_clean":"2024-05-14T17:03:37.683335+0000","last_became_active":"2024-05-14T17:00:10.054981+0000","last_became_peered":"2024-05-14T17:00:10.054981+0000","last_unstale":"2024-05-14T17:03:37.683335+0000","last_undegraded":"2024-05-14T17:03:37.683335+0000","last_fullsized":"2024-05-14T17:03:37.683335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:31.767528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789151+0000","last_change":"2024-05-14T17:00:10.060589+0000","last_active":"2024-05-14T17:03:37.789151+0000","last_peered":"2024-05-14T17:03:37.789151+0000","last_clean":"2024-05-14T17:03:37.789151+0000","last_became_active":"2024-05-14T17:00:10.059071+0000","last_became_peered":"2024-05-14T17:00:10.059071+0000","last_unstale":"2024-05-14T17:03:37.789151+0000","last_undegraded":"2024-05-14T17:03:37.789151+0000","last_fullsized":"2024-05-14T17:03:37.789151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:28:28.856599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.673941+0000","last_change":"2024-05-14T17:00:10.065352+0000","last_active":"2024-05-14T17:03:37.673941+0000","last_peered":"2024-05-14T17:03:37.673941+0000","last_clean":"2024-05-14T17:03:37.673941+0000","last_became_active":"2024-05-14T17:00:10.065047+0000","last_became_peered":"2024-05-14T17:00:10.065047+0000","last_unstale":"2024-05-14T17:03:37.673941+0000","last_undegraded":"2024-05-14T17:03:37.673941+0000","last_fullsized":"2024-05-14T17:03:37.673941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:18:32.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687162+0000","last_change":"2024-05-14T17:00:10.062178+0000","last_active":"2024-05-14T17:03:37.687162+0000","last_peered":"2024-05-14T17:03:37.687162+0000","last_clean":"2024-05-14T17:03:37.687162+0000","last_became_active":"2024-05-14T17:00:10.061820+0000","last_became_peered":"2024-05-14T17:00:10.061820+0000","last_unstale":"2024-05-14T17:03:37.687162+0000","last_undegraded":"2024-05-14T17:03:37.687162+0000","last_fullsized":"2024-05-14T17:03:37.687162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:19:40.145851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683580+0000","last_change":"2024-05-14T17:00:10.063141+0000","last_active":"2024-05-14T17:03:37.683580+0000","last_peered":"2024-05-14T17:03:37.683580+0000","last_clean":"2024-05-14T17:03:37.683580+0000","last_became_active":"2024-05-14T17:00:10.062773+0000","last_became_peered":"2024-05-14T17:00:10.062773+0000","last_unstale":"2024-05-14T17:03:37.683580+0000","last_undegraded":"2024-05-14T17:03:37.683580+0000","last_fullsized":"2024-05-14T17:03:37.683580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:28:37.775874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789385+0000","last_change":"2024-05-14T17:00:10.064498+0000","last_active":"2024-05-14T17:03:37.789385+0000","last_peered":"2024-05-14T17:03:37.789385+0000","last_clean":"2024-05-14T17:03:37.789385+0000","last_became_active":"2024-05-14T17:00:10.064132+0000","last_became_peered":"2024-05-14T17:00:10.064132+0000","last_unstale":"2024-05-14T17:03:37.789385+0000","last_undegraded":"2024-05-14T17:03:37.789385+0000","last_fullsized":"2024-05-14T17:03:37.789385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:50.186280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577660+0000","last_change":"2024-05-14T17:00:10.055953+0000","last_active":"2024-05-14T17:03:37.577660+0000","last_peered":"2024-05-14T17:03:37.577660+0000","last_clean":"2024-05-14T17:03:37.577660+0000","last_became_active":"2024-05-14T17:00:10.055582+0000","last_became_peered":"2024-05-14T17:00:10.055582+0000","last_unstale":"2024-05-14T17:03:37.577660+0000","last_undegraded":"2024-05-14T17:03:37.577660+0000","last_fullsized":"2024-05-14T17:03:37.577660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:23:25.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688782+0000","last_change":"2024-05-14T17:00:10.064350+0000","last_active":"2024-05-14T17:03:37.688782+0000","last_peered":"2024-05-14T17:03:37.688782+0000","last_clean":"2024-05-14T17:03:37.688782+0000","last_became_active":"2024-05-14T17:00:10.063035+0000","last_became_peered":"2024-05-14T17:00:10.063035+0000","last_unstale":"2024-05-14T17:03:37.688782+0000","last_undegraded":"2024-05-14T17:03:37.688782+0000","last_fullsized":"2024-05-14T17:03:37.688782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:25.751182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686967+0000","last_change":"2024-05-14T17:00:10.057547+0000","last_active":"2024-05-14T17:03:37.686967+0000","last_peered":"2024-05-14T17:03:37.686967+0000","last_clean":"2024-05-14T17:03:37.686967+0000","last_became_active":"2024-05-14T17:00:10.057130+0000","last_became_peered":"2024-05-14T17:00:10.057130+0000","last_unstale":"2024-05-14T17:03:37.686967+0000","last_undegraded":"2024-05-14T17:03:37.686967+0000","last_fullsized":"2024-05-14T17:03:37.686967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:48.803824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686877+0000","last_change":"2024-05-14T17:00:10.060300+0000","last_active":"2024-05-14T17:03:37.686877+0000","last_peered":"2024-05-14T17:03:37.686877+0000","last_clean":"2024-05-14T17:03:37.686877+0000","last_became_active":"2024-05-14T17:00:10.059597+0000","last_became_peered":"2024-05-14T17:00:10.059597+0000","last_unstale":"2024-05-14T17:03:37.686877+0000","last_undegraded":"2024-05-14T17:03:37.686877+0000","last_fullsized":"2024-05-14T17:03:37.686877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:45:40.919281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686703+0000","last_change":"2024-05-14T17:00:10.067323+0000","last_active":"2024-05-14T17:03:37.686703+0000","last_peered":"2024-05-14T17:03:37.686703+0000","last_clean":"2024-05-14T17:03:37.686703+0000","last_became_active":"2024-05-14T17:00:10.066484+0000","last_became_peered":"2024-05-14T17:00:10.066484+0000","last_unstale":"2024-05-14T17:03:37.686703+0000","last_undegraded":"2024-05-14T17:03:37.686703+0000","last_fullsized":"2024-05-14T17:03:37.686703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:25:04.268503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687437+0000","last_change":"2024-05-14T17:00:10.063903+0000","last_active":"2024-05-14T17:03:37.687437+0000","last_peered":"2024-05-14T17:03:37.687437+0000","last_clean":"2024-05-14T17:03:37.687437+0000","last_became_active":"2024-05-14T17:00:10.063264+0000","last_became_peered":"2024-05-14T17:00:10.063264+0000","last_unstale":"2024-05-14T17:03:37.687437+0000","last_undegraded":"2024-05-14T17:03:37.687437+0000","last_fullsized":"2024-05-14T17:03:37.687437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:37:43.392210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573578+0000","last_change":"2024-05-14T17:00:10.058647+0000","last_active":"2024-05-14T17:03:37.573578+0000","last_peered":"2024-05-14T17:03:37.573578+0000","last_clean":"2024-05-14T17:03:37.573578+0000","last_became_active":"2024-05-14T17:00:10.058312+0000","last_became_peered":"2024-05-14T17:00:10.058312+0000","last_unstale":"2024-05-14T17:03:37.573578+0000","last_undegraded":"2024-05-14T17:03:37.573578+0000","last_fullsized":"2024-05-14T17:03:37.573578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:25.055566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575796+0000","last_change":"2024-05-14T17:00:10.064766+0000","last_active":"2024-05-14T17:03:37.575796+0000","last_peered":"2024-05-14T17:03:37.575796+0000","last_clean":"2024-05-14T17:03:37.575796+0000","last_became_active":"2024-05-14T17:00:10.064419+0000","last_became_peered":"2024-05-14T17:00:10.064419+0000","last_unstale":"2024-05-14T17:03:37.575796+0000","last_undegraded":"2024-05-14T17:03:37.575796+0000","last_fullsized":"2024-05-14T17:03:37.575796+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:09.213153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573421+0000","last_change":"2024-05-14T17:00:07.040250+0000","last_active":"2024-05-14T17:03:37.573421+0000","last_peered":"2024-05-14T17:03:37.573421+0000","last_clean":"2024-05-14T17:03:37.573421+0000","last_became_active":"2024-05-14T17:00:07.039718+0000","last_became_peered":"2024-05-14T17:00:07.039718+0000","last_unstale":"2024-05-14T17:03:37.573421+0000","last_undegraded":"2024-05-14T17:03:37.573421+0000","last_fullsized":"2024-05-14T17:03:37.573421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:17:54.686862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674758+0000","last_change":"2024-05-14T17:00:10.070393+0000","last_active":"2024-05-14T17:03:37.674758+0000","last_peered":"2024-05-14T17:03:37.674758+0000","last_clean":"2024-05-14T17:03:37.674758+0000","last_became_active":"2024-05-14T17:00:10.069455+0000","last_became_peered":"2024-05-14T17:00:10.069455+0000","last_unstale":"2024-05-14T17:03:37.674758+0000","last_undegraded":"2024-05-14T17:03:37.674758+0000","last_fullsized":"2024-05-14T17:03:37.674758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:12.637942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674912+0000","last_change":"2024-05-14T17:00:07.162609+0000","last_active":"2024-05-14T17:03:37.674912+0000","last_peered":"2024-05-14T17:03:37.674912+0000","last_clean":"2024-05-14T17:03:37.674912+0000","last_became_active":"2024-05-14T17:00:07.162029+0000","last_became_peered":"2024-05-14T17:00:07.162029+0000","last_unstale":"2024-05-14T17:03:37.674912+0000","last_undegraded":"2024-05-14T17:03:37.674912+0000","last_fullsized":"2024-05-14T17:03:37.674912+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:13:56.089807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687882+0000","last_change":"2024-05-14T17:00:10.061421+0000","last_active":"2024-05-14T17:03:37.687882+0000","last_peered":"2024-05-14T17:03:37.687882+0000","last_clean":"2024-05-14T17:03:37.687882+0000","last_became_active":"2024-05-14T17:00:10.061072+0000","last_became_peered":"2024-05-14T17:00:10.061072+0000","last_unstale":"2024-05-14T17:03:37.687882+0000","last_undegraded":"2024-05-14T17:03:37.687882+0000","last_fullsized":"2024-05-14T17:03:37.687882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:01.816303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674597+0000","last_change":"2024-05-14T17:00:07.043687+0000","last_active":"2024-05-14T17:03:37.674597+0000","last_peered":"2024-05-14T17:03:37.674597+0000","last_clean":"2024-05-14T17:03:37.674597+0000","last_became_active":"2024-05-14T17:00:07.043314+0000","last_became_peered":"2024-05-14T17:00:07.043314+0000","last_unstale":"2024-05-14T17:03:37.674597+0000","last_undegraded":"2024-05-14T17:03:37.674597+0000","last_fullsized":"2024-05-14T17:03:37.674597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:45:45.850295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686859+0000","last_change":"2024-05-14T17:00:10.063918+0000","last_active":"2024-05-14T17:03:37.686859+0000","last_peered":"2024-05-14T17:03:37.686859+0000","last_clean":"2024-05-14T17:03:37.686859+0000","last_became_active":"2024-05-14T17:00:10.063526+0000","last_became_peered":"2024-05-14T17:00:10.063526+0000","last_unstale":"2024-05-14T17:03:37.686859+0000","last_undegraded":"2024-05-14T17:03:37.686859+0000","last_fullsized":"2024-05-14T17:03:37.686859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:31:15.750326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674465+0000","last_change":"2024-05-14T17:00:07.040181+0000","last_active":"2024-05-14T17:03:37.674465+0000","last_peered":"2024-05-14T17:03:37.674465+0000","last_clean":"2024-05-14T17:03:37.674465+0000","last_became_active":"2024-05-14T17:00:07.039781+0000","last_became_peered":"2024-05-14T17:00:07.039781+0000","last_unstale":"2024-05-14T17:03:37.674465+0000","last_undegraded":"2024-05-14T17:03:37.674465+0000","last_fullsized":"2024-05-14T17:03:37.674465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:04:04.037371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680651+0000","last_change":"2024-05-14T17:00:10.058411+0000","last_active":"2024-05-14T17:03:37.680651+0000","last_peered":"2024-05-14T17:03:37.680651+0000","last_clean":"2024-05-14T17:03:37.680651+0000","last_became_active":"2024-05-14T17:00:10.058063+0000","last_became_peered":"2024-05-14T17:00:10.058063+0000","last_unstale":"2024-05-14T17:03:37.680651+0000","last_undegraded":"2024-05-14T17:03:37.680651+0000","last_fullsized":"2024-05-14T17:03:37.680651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:23:20.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":266,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:57.548684+0000","last_change":"2024-05-14T17:01:04.271215+0000","last_active":"2024-05-14T17:03:57.548684+0000","last_peered":"2024-05-14T17:03:57.548684+0000","last_clean":"2024-05-14T17:03:57.548684+0000","last_became_active":"2024-05-14T17:01:04.267186+0000","last_became_peered":"2024-05-14T17:01:04.267186+0000","last_unstale":"2024-05-14T17:03:57.548684+0000","last_undegraded":"2024-05-14T17:03:57.548684+0000","last_fullsized":"2024-05-14T17:03:57.548684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:01.465730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674220+0000","last_change":"2024-05-14T17:01:04.241079+0000","last_active":"2024-05-14T17:03:37.674220+0000","last_peered":"2024-05-14T17:03:37.674220+0000","last_clean":"2024-05-14T17:03:37.674220+0000","last_became_active":"2024-05-14T17:01:04.240698+0000","last_became_peered":"2024-05-14T17:01:04.240698+0000","last_unstale":"2024-05-14T17:03:37.674220+0000","last_undegraded":"2024-05-14T17:03:37.674220+0000","last_fullsized":"2024-05-14T17:03:37.674220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:11:10.589991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683244+0000","last_change":"2024-05-14T17:00:07.042657+0000","last_active":"2024-05-14T17:03:37.683244+0000","last_peered":"2024-05-14T17:03:37.683244+0000","last_clean":"2024-05-14T17:03:37.683244+0000","last_became_active":"2024-05-14T17:00:07.042287+0000","last_became_peered":"2024-05-14T17:00:07.042287+0000","last_unstale":"2024-05-14T17:03:37.683244+0000","last_undegraded":"2024-05-14T17:03:37.683244+0000","last_fullsized":"2024-05-14T17:03:37.683244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:02:44.091222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680431+0000","last_change":"2024-05-14T17:00:10.060889+0000","last_active":"2024-05-14T17:03:37.680431+0000","last_peered":"2024-05-14T17:03:37.680431+0000","last_clean":"2024-05-14T17:03:37.680431+0000","last_became_active":"2024-05-14T17:00:10.060466+0000","last_became_peered":"2024-05-14T17:00:10.060466+0000","last_unstale":"2024-05-14T17:03:37.680431+0000","last_undegraded":"2024-05-14T17:03:37.680431+0000","last_fullsized":"2024-05-14T17:03:37.680431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:49:35.237037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687440+0000","last_change":"2024-05-14T17:00:07.031391+0000","last_active":"2024-05-14T17:03:37.687440+0000","last_peered":"2024-05-14T17:03:37.687440+0000","last_clean":"2024-05-14T17:03:37.687440+0000","last_became_active":"2024-05-14T17:00:07.030990+0000","last_became_peered":"2024-05-14T17:00:07.030990+0000","last_unstale":"2024-05-14T17:03:37.687440+0000","last_undegraded":"2024-05-14T17:03:37.687440+0000","last_fullsized":"2024-05-14T17:03:37.687440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:39.418010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573316+0000","last_change":"2024-05-14T17:01:04.236554+0000","last_active":"2024-05-14T17:03:37.573316+0000","last_peered":"2024-05-14T17:03:37.573316+0000","last_clean":"2024-05-14T17:03:37.573316+0000","last_became_active":"2024-05-14T17:01:04.236203+0000","last_became_peered":"2024-05-14T17:01:04.236203+0000","last_unstale":"2024-05-14T17:03:37.573316+0000","last_undegraded":"2024-05-14T17:03:37.573316+0000","last_fullsized":"2024-05-14T17:03:37.573316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:34.421246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680247+0000","last_change":"2024-05-14T17:00:07.037209+0000","last_active":"2024-05-14T17:03:37.680247+0000","last_peered":"2024-05-14T17:03:37.680247+0000","last_clean":"2024-05-14T17:03:37.680247+0000","last_became_active":"2024-05-14T17:00:07.036615+0000","last_became_peered":"2024-05-14T17:00:07.036615+0000","last_unstale":"2024-05-14T17:03:37.680247+0000","last_undegraded":"2024-05-14T17:03:37.680247+0000","last_fullsized":"2024-05-14T17:03:37.680247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:05:20.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573284+0000","last_change":"2024-05-14T17:00:10.057189+0000","last_active":"2024-05-14T17:03:37.573284+0000","last_peered":"2024-05-14T17:03:37.573284+0000","last_clean":"2024-05-14T17:03:37.573284+0000","last_became_active":"2024-05-14T17:00:10.056834+0000","last_became_peered":"2024-05-14T17:00:10.056834+0000","last_unstale":"2024-05-14T17:03:37.573284+0000","last_undegraded":"2024-05-14T17:03:37.573284+0000","last_fullsized":"2024-05-14T17:03:37.573284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:02:55.698720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":245,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:57.548017+0000","last_change":"2024-05-14T17:00:07.039130+0000","last_active":"2024-05-14T17:03:57.548017+0000","last_peered":"2024-05-14T17:03:57.548017+0000","last_clean":"2024-05-14T17:03:57.548017+0000","last_became_active":"2024-05-14T17:00:07.038688+0000","last_became_peered":"2024-05-14T17:00:07.038688+0000","last_unstale":"2024-05-14T17:03:57.548017+0000","last_undegraded":"2024-05-14T17:03:57.548017+0000","last_fullsized":"2024-05-14T17:03:57.548017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:18:57.629966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":211,"num_read_kb":211,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687113+0000","last_change":"2024-05-14T17:00:10.058692+0000","last_active":"2024-05-14T17:03:37.687113+0000","last_peered":"2024-05-14T17:03:37.687113+0000","last_clean":"2024-05-14T17:03:37.687113+0000","last_became_active":"2024-05-14T17:00:10.058292+0000","last_became_peered":"2024-05-14T17:00:10.058292+0000","last_unstale":"2024-05-14T17:03:37.687113+0000","last_undegraded":"2024-05-14T17:03:37.687113+0000","last_fullsized":"2024-05-14T17:03:37.687113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:02.627323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789349+0000","last_change":"2024-05-14T17:00:07.038596+0000","last_active":"2024-05-14T17:03:37.789349+0000","last_peered":"2024-05-14T17:03:37.789349+0000","last_clean":"2024-05-14T17:03:37.789349+0000","last_became_active":"2024-05-14T17:00:07.038192+0000","last_became_peered":"2024-05-14T17:00:07.038192+0000","last_unstale":"2024-05-14T17:03:37.789349+0000","last_undegraded":"2024-05-14T17:03:37.789349+0000","last_fullsized":"2024-05-14T17:03:37.789349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:48:44.641415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687808+0000","last_change":"2024-05-14T17:00:10.054717+0000","last_active":"2024-05-14T17:03:37.687808+0000","last_peered":"2024-05-14T17:03:37.687808+0000","last_clean":"2024-05-14T17:03:37.687808+0000","last_became_active":"2024-05-14T17:00:10.054386+0000","last_became_peered":"2024-05-14T17:00:10.054386+0000","last_unstale":"2024-05-14T17:03:37.687808+0000","last_undegraded":"2024-05-14T17:03:37.687808+0000","last_fullsized":"2024-05-14T17:03:37.687808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:34:59.097755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680791+0000","last_change":"2024-05-14T17:00:07.163188+0000","last_active":"2024-05-14T17:03:37.680791+0000","last_peered":"2024-05-14T17:03:37.680791+0000","last_clean":"2024-05-14T17:03:37.680791+0000","last_became_active":"2024-05-14T17:00:07.162663+0000","last_became_peered":"2024-05-14T17:00:07.162663+0000","last_unstale":"2024-05-14T17:03:37.680791+0000","last_undegraded":"2024-05-14T17:03:37.680791+0000","last_fullsized":"2024-05-14T17:03:37.680791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:46:29.151774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686928+0000","last_change":"2024-05-14T17:00:10.061365+0000","last_active":"2024-05-14T17:03:37.686928+0000","last_peered":"2024-05-14T17:03:37.686928+0000","last_clean":"2024-05-14T17:03:37.686928+0000","last_became_active":"2024-05-14T17:00:10.061056+0000","last_became_peered":"2024-05-14T17:00:10.061056+0000","last_unstale":"2024-05-14T17:03:37.686928+0000","last_undegraded":"2024-05-14T17:03:37.686928+0000","last_fullsized":"2024-05-14T17:03:37.686928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:44:17.788717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":498,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:58.549827+0000","last_change":"2024-05-14T17:00:07.041674+0000","last_active":"2024-05-14T17:03:58.549827+0000","last_peered":"2024-05-14T17:03:58.549827+0000","last_clean":"2024-05-14T17:03:58.549827+0000","last_became_active":"2024-05-14T17:00:07.041044+0000","last_became_peered":"2024-05-14T17:00:07.041044+0000","last_unstale":"2024-05-14T17:03:58.549827+0000","last_undegraded":"2024-05-14T17:03:58.549827+0000","last_fullsized":"2024-05-14T17:03:58.549827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:20.348882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":434,"num_read_kb":283,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674413+0000","last_change":"2024-05-14T17:00:10.064167+0000","last_active":"2024-05-14T17:03:37.674413+0000","last_peered":"2024-05-14T17:03:37.674413+0000","last_clean":"2024-05-14T17:03:37.674413+0000","last_became_active":"2024-05-14T17:00:10.063628+0000","last_became_peered":"2024-05-14T17:00:10.063628+0000","last_unstale":"2024-05-14T17:03:37.674413+0000","last_undegraded":"2024-05-14T17:03:37.674413+0000","last_fullsized":"2024-05-14T17:03:37.674413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:38.574848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687117+0000","last_change":"2024-05-14T17:01:04.249537+0000","last_active":"2024-05-14T17:03:37.687117+0000","last_peered":"2024-05-14T17:03:37.687117+0000","last_clean":"2024-05-14T17:03:37.687117+0000","last_became_active":"2024-05-14T17:01:04.239963+0000","last_became_peered":"2024-05-14T17:01:04.239963+0000","last_unstale":"2024-05-14T17:03:37.687117+0000","last_undegraded":"2024-05-14T17:03:37.687117+0000","last_fullsized":"2024-05-14T17:03:37.687117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:00:51.798116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688287+0000","last_change":"2024-05-14T17:00:10.065719+0000","last_active":"2024-05-14T17:03:37.688287+0000","last_peered":"2024-05-14T17:03:37.688287+0000","last_clean":"2024-05-14T17:03:37.688287+0000","last_became_active":"2024-05-14T17:00:10.065433+0000","last_became_peered":"2024-05-14T17:00:10.065433+0000","last_unstale":"2024-05-14T17:03:37.688287+0000","last_undegraded":"2024-05-14T17:03:37.688287+0000","last_fullsized":"2024-05-14T17:03:37.688287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:16.974580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680371+0000","last_change":"2024-05-14T17:00:07.038975+0000","last_active":"2024-05-14T17:03:37.680371+0000","last_peered":"2024-05-14T17:03:37.680371+0000","last_clean":"2024-05-14T17:03:37.680371+0000","last_became_active":"2024-05-14T17:00:07.038614+0000","last_became_peered":"2024-05-14T17:00:07.038614+0000","last_unstale":"2024-05-14T17:03:37.680371+0000","last_undegraded":"2024-05-14T17:03:37.680371+0000","last_fullsized":"2024-05-14T17:03:37.680371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:52:29.335366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789348+0000","last_change":"2024-05-14T17:00:10.060870+0000","last_active":"2024-05-14T17:03:37.789348+0000","last_peered":"2024-05-14T17:03:37.789348+0000","last_clean":"2024-05-14T17:03:37.789348+0000","last_became_active":"2024-05-14T17:00:10.060356+0000","last_became_peered":"2024-05-14T17:00:10.060356+0000","last_unstale":"2024-05-14T17:03:37.789348+0000","last_undegraded":"2024-05-14T17:03:37.789348+0000","last_fullsized":"2024-05-14T17:03:37.789348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:33:36.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":247,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:57.455615+0000","last_change":"2024-05-14T17:00:07.033732+0000","last_active":"2024-05-14T17:03:57.455615+0000","last_peered":"2024-05-14T17:03:57.455615+0000","last_clean":"2024-05-14T17:03:57.455615+0000","last_became_active":"2024-05-14T17:00:07.033153+0000","last_became_peered":"2024-05-14T17:00:07.033153+0000","last_unstale":"2024-05-14T17:03:57.455615+0000","last_undegraded":"2024-05-14T17:03:57.455615+0000","last_fullsized":"2024-05-14T17:03:57.455615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:01.675276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789088+0000","last_change":"2024-05-14T17:00:10.062649+0000","last_active":"2024-05-14T17:03:37.789088+0000","last_peered":"2024-05-14T17:03:37.789088+0000","last_clean":"2024-05-14T17:03:37.789088+0000","last_became_active":"2024-05-14T17:00:10.061986+0000","last_became_peered":"2024-05-14T17:00:10.061986+0000","last_unstale":"2024-05-14T17:03:37.789088+0000","last_undegraded":"2024-05-14T17:03:37.789088+0000","last_fullsized":"2024-05-14T17:03:37.789088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:07:46.196349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674198+0000","last_change":"2024-05-14T17:00:07.163514+0000","last_active":"2024-05-14T17:03:37.674198+0000","last_peered":"2024-05-14T17:03:37.674198+0000","last_clean":"2024-05-14T17:03:37.674198+0000","last_became_active":"2024-05-14T17:00:07.163034+0000","last_became_peered":"2024-05-14T17:00:07.163034+0000","last_unstale":"2024-05-14T17:03:37.674198+0000","last_undegraded":"2024-05-14T17:03:37.674198+0000","last_fullsized":"2024-05-14T17:03:37.674198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:43:28.322554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680298+0000","last_change":"2024-05-14T17:00:10.056916+0000","last_active":"2024-05-14T17:03:37.680298+0000","last_peered":"2024-05-14T17:03:37.680298+0000","last_clean":"2024-05-14T17:03:37.680298+0000","last_became_active":"2024-05-14T17:00:10.056558+0000","last_became_peered":"2024-05-14T17:00:10.056558+0000","last_unstale":"2024-05-14T17:03:37.680298+0000","last_undegraded":"2024-05-14T17:03:37.680298+0000","last_fullsized":"2024-05-14T17:03:37.680298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:02:52.173662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674493+0000","last_change":"2024-05-14T17:00:07.042532+0000","last_active":"2024-05-14T17:03:37.674493+0000","last_peered":"2024-05-14T17:03:37.674493+0000","last_clean":"2024-05-14T17:03:37.674493+0000","last_became_active":"2024-05-14T17:00:07.042192+0000","last_became_peered":"2024-05-14T17:00:07.042192+0000","last_unstale":"2024-05-14T17:03:37.674493+0000","last_undegraded":"2024-05-14T17:03:37.674493+0000","last_fullsized":"2024-05-14T17:03:37.674493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:31:25.355063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687257+0000","last_change":"2024-05-14T17:00:10.061438+0000","last_active":"2024-05-14T17:03:37.687257+0000","last_peered":"2024-05-14T17:03:37.687257+0000","last_clean":"2024-05-14T17:03:37.687257+0000","last_became_active":"2024-05-14T17:00:10.057548+0000","last_became_peered":"2024-05-14T17:00:10.057548+0000","last_unstale":"2024-05-14T17:03:37.687257+0000","last_undegraded":"2024-05-14T17:03:37.687257+0000","last_fullsized":"2024-05-14T17:03:37.687257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:58.533836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573679+0000","last_change":"2024-05-14T17:00:07.163775+0000","last_active":"2024-05-14T17:03:37.573679+0000","last_peered":"2024-05-14T17:03:37.573679+0000","last_clean":"2024-05-14T17:03:37.573679+0000","last_became_active":"2024-05-14T17:00:07.163468+0000","last_became_peered":"2024-05-14T17:00:07.163468+0000","last_unstale":"2024-05-14T17:03:37.573679+0000","last_undegraded":"2024-05-14T17:03:37.573679+0000","last_fullsized":"2024-05-14T17:03:37.573679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:27.580924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683414+0000","last_change":"2024-05-14T17:00:10.058741+0000","last_active":"2024-05-14T17:03:37.683414+0000","last_peered":"2024-05-14T17:03:37.683414+0000","last_clean":"2024-05-14T17:03:37.683414+0000","last_became_active":"2024-05-14T17:00:10.058375+0000","last_became_peered":"2024-05-14T17:00:10.058375+0000","last_unstale":"2024-05-14T17:03:37.683414+0000","last_undegraded":"2024-05-14T17:03:37.683414+0000","last_fullsized":"2024-05-14T17:03:37.683414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:23.477248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789470+0000","last_change":"2024-05-14T17:00:07.033098+0000","last_active":"2024-05-14T17:03:37.789470+0000","last_peered":"2024-05-14T17:03:37.789470+0000","last_clean":"2024-05-14T17:03:37.789470+0000","last_became_active":"2024-05-14T17:00:07.032746+0000","last_became_peered":"2024-05-14T17:00:07.032746+0000","last_unstale":"2024-05-14T17:03:37.789470+0000","last_undegraded":"2024-05-14T17:03:37.789470+0000","last_fullsized":"2024-05-14T17:03:37.789470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:07:47.950011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.683335+0000","last_change":"2024-05-14T17:00:10.056600+0000","last_active":"2024-05-14T17:03:37.683335+0000","last_peered":"2024-05-14T17:03:37.683335+0000","last_clean":"2024-05-14T17:03:37.683335+0000","last_became_active":"2024-05-14T17:00:10.056244+0000","last_became_peered":"2024-05-14T17:00:10.056244+0000","last_unstale":"2024-05-14T17:03:37.683335+0000","last_undegraded":"2024-05-14T17:03:37.683335+0000","last_fullsized":"2024-05-14T17:03:37.683335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:57:43.602324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.573215+0000","last_change":"2024-05-14T17:01:04.237414+0000","last_active":"2024-05-14T17:03:37.573215+0000","last_peered":"2024-05-14T17:03:37.573215+0000","last_clean":"2024-05-14T17:03:37.573215+0000","last_became_active":"2024-05-14T17:01:04.237043+0000","last_became_peered":"2024-05-14T17:01:04.237043+0000","last_unstale":"2024-05-14T17:03:37.573215+0000","last_undegraded":"2024-05-14T17:03:37.573215+0000","last_fullsized":"2024-05-14T17:03:37.573215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:15:33.303302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":245,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:57.455038+0000","last_change":"2024-05-14T17:00:07.162347+0000","last_active":"2024-05-14T17:03:57.455038+0000","last_peered":"2024-05-14T17:03:57.455038+0000","last_clean":"2024-05-14T17:03:57.455038+0000","last_became_active":"2024-05-14T17:00:07.162015+0000","last_became_peered":"2024-05-14T17:00:07.162015+0000","last_unstale":"2024-05-14T17:03:57.455038+0000","last_undegraded":"2024-05-14T17:03:57.455038+0000","last_fullsized":"2024-05-14T17:03:57.455038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:05:45.070617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":211,"num_read_kb":211,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674179+0000","last_change":"2024-05-14T17:00:10.065917+0000","last_active":"2024-05-14T17:03:37.674179+0000","last_peered":"2024-05-14T17:03:37.674179+0000","last_clean":"2024-05-14T17:03:37.674179+0000","last_became_active":"2024-05-14T17:00:10.065612+0000","last_became_peered":"2024-05-14T17:00:10.065612+0000","last_unstale":"2024-05-14T17:03:37.674179+0000","last_undegraded":"2024-05-14T17:03:37.674179+0000","last_fullsized":"2024-05-14T17:03:37.674179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:36:20.904300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577611+0000","last_change":"2024-05-14T17:00:07.032697+0000","last_active":"2024-05-14T17:03:37.577611+0000","last_peered":"2024-05-14T17:03:37.577611+0000","last_clean":"2024-05-14T17:03:37.577611+0000","last_became_active":"2024-05-14T17:00:07.032246+0000","last_became_peered":"2024-05-14T17:00:07.032246+0000","last_unstale":"2024-05-14T17:03:37.577611+0000","last_undegraded":"2024-05-14T17:03:37.577611+0000","last_fullsized":"2024-05-14T17:03:37.577611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:14:06.896887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:39.675244+0000","last_change":"2024-05-14T16:58:40.948814+0000","last_active":"2024-05-14T17:03:39.675244+0000","last_peered":"2024-05-14T17:03:39.675244+0000","last_clean":"2024-05-14T17:03:39.675244+0000","last_became_active":"2024-05-14T16:58:40.942917+0000","last_became_peered":"2024-05-14T16:58:40.942917+0000","last_unstale":"2024-05-14T17:03:39.675244+0000","last_undegraded":"2024-05-14T17:03:39.675244+0000","last_fullsized":"2024-05-14T17:03:39.675244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:55:27.007968+0000","last_clean_scrub_stamp":"2024-05-14T16:55:27.007968+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:14:48.146762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687172+0000","last_change":"2024-05-14T17:00:10.063657+0000","last_active":"2024-05-14T17:03:37.687172+0000","last_peered":"2024-05-14T17:03:37.687172+0000","last_clean":"2024-05-14T17:03:37.687172+0000","last_became_active":"2024-05-14T17:00:10.063357+0000","last_became_peered":"2024-05-14T17:00:10.063357+0000","last_unstale":"2024-05-14T17:03:37.687172+0000","last_undegraded":"2024-05-14T17:03:37.687172+0000","last_fullsized":"2024-05-14T17:03:37.687172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:51:27.659902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789348+0000","last_change":"2024-05-14T17:00:07.040052+0000","last_active":"2024-05-14T17:03:37.789348+0000","last_peered":"2024-05-14T17:03:37.789348+0000","last_clean":"2024-05-14T17:03:37.789348+0000","last_became_active":"2024-05-14T17:00:07.039697+0000","last_became_peered":"2024-05-14T17:00:07.039697+0000","last_unstale":"2024-05-14T17:03:37.789348+0000","last_undegraded":"2024-05-14T17:03:37.789348+0000","last_fullsized":"2024-05-14T17:03:37.789348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:35:36.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674457+0000","last_change":"2024-05-14T17:00:07.162468+0000","last_active":"2024-05-14T17:03:37.674457+0000","last_peered":"2024-05-14T17:03:37.674457+0000","last_clean":"2024-05-14T17:03:37.674457+0000","last_became_active":"2024-05-14T17:00:07.162071+0000","last_became_peered":"2024-05-14T17:00:07.162071+0000","last_unstale":"2024-05-14T17:03:37.674457+0000","last_undegraded":"2024-05-14T17:03:37.674457+0000","last_fullsized":"2024-05-14T17:03:37.674457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:45:51.138133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688217+0000","last_change":"2024-05-14T17:00:10.063809+0000","last_active":"2024-05-14T17:03:37.688217+0000","last_peered":"2024-05-14T17:03:37.688217+0000","last_clean":"2024-05-14T17:03:37.688217+0000","last_became_active":"2024-05-14T17:00:10.063411+0000","last_became_peered":"2024-05-14T17:00:10.063411+0000","last_unstale":"2024-05-14T17:03:37.688217+0000","last_undegraded":"2024-05-14T17:03:37.688217+0000","last_fullsized":"2024-05-14T17:03:37.688217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:09.180600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'100","reported_seq":221,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:53.281414+0000","last_change":"2024-05-14T17:00:07.040099+0000","last_active":"2024-05-14T17:03:53.281414+0000","last_peered":"2024-05-14T17:03:53.281414+0000","last_clean":"2024-05-14T17:03:53.281414+0000","last_became_active":"2024-05-14T17:00:07.039621+0000","last_became_peered":"2024-05-14T17:00:07.039621+0000","last_unstale":"2024-05-14T17:03:53.281414+0000","last_undegraded":"2024-05-14T17:03:53.281414+0000","last_fullsized":"2024-05-14T17:03:53.281414+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:25:04.185854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":184,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674648+0000","last_change":"2024-05-14T17:00:10.067195+0000","last_active":"2024-05-14T17:03:37.674648+0000","last_peered":"2024-05-14T17:03:37.674648+0000","last_clean":"2024-05-14T17:03:37.674648+0000","last_became_active":"2024-05-14T17:00:10.066883+0000","last_became_peered":"2024-05-14T17:00:10.066883+0000","last_unstale":"2024-05-14T17:03:37.674648+0000","last_undegraded":"2024-05-14T17:03:37.674648+0000","last_fullsized":"2024-05-14T17:03:37.674648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:55:09.580092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.577683+0000","last_change":"2024-05-14T17:00:07.031974+0000","last_active":"2024-05-14T17:03:37.577683+0000","last_peered":"2024-05-14T17:03:37.577683+0000","last_clean":"2024-05-14T17:03:37.577683+0000","last_became_active":"2024-05-14T17:00:07.031495+0000","last_became_peered":"2024-05-14T17:00:07.031495+0000","last_unstale":"2024-05-14T17:03:37.577683+0000","last_undegraded":"2024-05-14T17:03:37.577683+0000","last_fullsized":"2024-05-14T17:03:37.577683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:54:55.094403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687669+0000","last_change":"2024-05-14T17:00:10.071212+0000","last_active":"2024-05-14T17:03:37.687669+0000","last_peered":"2024-05-14T17:03:37.687669+0000","last_clean":"2024-05-14T17:03:37.687669+0000","last_became_active":"2024-05-14T17:00:10.070837+0000","last_became_peered":"2024-05-14T17:00:10.070837+0000","last_unstale":"2024-05-14T17:03:37.687669+0000","last_undegraded":"2024-05-14T17:03:37.687669+0000","last_fullsized":"2024-05-14T17:03:37.687669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:48:38.218757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686711+0000","last_change":"2024-05-14T17:00:07.163356+0000","last_active":"2024-05-14T17:03:37.686711+0000","last_peered":"2024-05-14T17:03:37.686711+0000","last_clean":"2024-05-14T17:03:37.686711+0000","last_became_active":"2024-05-14T17:00:07.162936+0000","last_became_peered":"2024-05-14T17:00:07.162936+0000","last_unstale":"2024-05-14T17:03:37.686711+0000","last_undegraded":"2024-05-14T17:03:37.686711+0000","last_fullsized":"2024-05-14T17:03:37.686711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:43.565047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674192+0000","last_change":"2024-05-14T17:00:10.059641+0000","last_active":"2024-05-14T17:03:37.674192+0000","last_peered":"2024-05-14T17:03:37.674192+0000","last_clean":"2024-05-14T17:03:37.674192+0000","last_became_active":"2024-05-14T17:00:10.058980+0000","last_became_peered":"2024-05-14T17:00:10.058980+0000","last_unstale":"2024-05-14T17:03:37.674192+0000","last_undegraded":"2024-05-14T17:03:37.674192+0000","last_fullsized":"2024-05-14T17:03:37.674192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:15:21.630452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674032+0000","last_change":"2024-05-14T17:00:07.032086+0000","last_active":"2024-05-14T17:03:37.674032+0000","last_peered":"2024-05-14T17:03:37.674032+0000","last_clean":"2024-05-14T17:03:37.674032+0000","last_became_active":"2024-05-14T17:00:07.031697+0000","last_became_peered":"2024-05-14T17:00:07.031697+0000","last_unstale":"2024-05-14T17:03:37.674032+0000","last_undegraded":"2024-05-14T17:03:37.674032+0000","last_fullsized":"2024-05-14T17:03:37.674032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:57:00.150935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686985+0000","last_change":"2024-05-14T17:00:10.066679+0000","last_active":"2024-05-14T17:03:37.686985+0000","last_peered":"2024-05-14T17:03:37.686985+0000","last_clean":"2024-05-14T17:03:37.686985+0000","last_became_active":"2024-05-14T17:00:10.065857+0000","last_became_peered":"2024-05-14T17:00:10.065857+0000","last_unstale":"2024-05-14T17:03:37.686985+0000","last_undegraded":"2024-05-14T17:03:37.686985+0000","last_fullsized":"2024-05-14T17:03:37.686985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:58.848284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575638+0000","last_change":"2024-05-14T17:00:07.042520+0000","last_active":"2024-05-14T17:03:37.575638+0000","last_peered":"2024-05-14T17:03:37.575638+0000","last_clean":"2024-05-14T17:03:37.575638+0000","last_became_active":"2024-05-14T17:00:07.042099+0000","last_became_peered":"2024-05-14T17:00:07.042099+0000","last_unstale":"2024-05-14T17:03:37.575638+0000","last_undegraded":"2024-05-14T17:03:37.575638+0000","last_fullsized":"2024-05-14T17:03:37.575638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:39:38.920335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674360+0000","last_change":"2024-05-14T17:00:10.064297+0000","last_active":"2024-05-14T17:03:37.674360+0000","last_peered":"2024-05-14T17:03:37.674360+0000","last_clean":"2024-05-14T17:03:37.674360+0000","last_became_active":"2024-05-14T17:00:10.064000+0000","last_became_peered":"2024-05-14T17:00:10.064000+0000","last_unstale":"2024-05-14T17:03:37.674360+0000","last_undegraded":"2024-05-14T17:03:37.674360+0000","last_fullsized":"2024-05-14T17:03:37.674360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:36:20.433817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687026+0000","last_change":"2024-05-14T17:01:04.243512+0000","last_active":"2024-05-14T17:03:37.687026+0000","last_peered":"2024-05-14T17:03:37.687026+0000","last_clean":"2024-05-14T17:03:37.687026+0000","last_became_active":"2024-05-14T17:01:04.237118+0000","last_became_peered":"2024-05-14T17:01:04.237118+0000","last_unstale":"2024-05-14T17:03:37.687026+0000","last_undegraded":"2024-05-14T17:03:37.687026+0000","last_fullsized":"2024-05-14T17:03:37.687026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:19:16.524481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789265+0000","last_change":"2024-05-14T17:00:10.065528+0000","last_active":"2024-05-14T17:03:37.789265+0000","last_peered":"2024-05-14T17:03:37.789265+0000","last_clean":"2024-05-14T17:03:37.789265+0000","last_became_active":"2024-05-14T17:00:10.065202+0000","last_became_peered":"2024-05-14T17:00:10.065202+0000","last_unstale":"2024-05-14T17:03:37.789265+0000","last_undegraded":"2024-05-14T17:03:37.789265+0000","last_fullsized":"2024-05-14T17:03:37.789265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:35:10.878326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575766+0000","last_change":"2024-05-14T17:00:07.163678+0000","last_active":"2024-05-14T17:03:37.575766+0000","last_peered":"2024-05-14T17:03:37.575766+0000","last_clean":"2024-05-14T17:03:37.575766+0000","last_became_active":"2024-05-14T17:00:07.163189+0000","last_became_peered":"2024-05-14T17:00:07.163189+0000","last_unstale":"2024-05-14T17:03:37.575766+0000","last_undegraded":"2024-05-14T17:03:37.575766+0000","last_fullsized":"2024-05-14T17:03:37.575766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:55:33.079644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789479+0000","last_change":"2024-05-14T17:00:10.063699+0000","last_active":"2024-05-14T17:03:37.789479+0000","last_peered":"2024-05-14T17:03:37.789479+0000","last_clean":"2024-05-14T17:03:37.789479+0000","last_became_active":"2024-05-14T17:00:10.063356+0000","last_became_peered":"2024-05-14T17:00:10.063356+0000","last_unstale":"2024-05-14T17:03:37.789479+0000","last_undegraded":"2024-05-14T17:03:37.789479+0000","last_fullsized":"2024-05-14T17:03:37.789479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:01:20.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'65","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:52.433348+0000","last_change":"2024-05-14T17:00:07.037470+0000","last_active":"2024-05-14T17:03:52.433348+0000","last_peered":"2024-05-14T17:03:52.433348+0000","last_clean":"2024-05-14T17:03:52.433348+0000","last_became_active":"2024-05-14T17:00:07.037129+0000","last_became_peered":"2024-05-14T17:00:07.037129+0000","last_unstale":"2024-05-14T17:03:52.433348+0000","last_undegraded":"2024-05-14T17:03:52.433348+0000","last_fullsized":"2024-05-14T17:03:52.433348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:01.552999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789250+0000","last_change":"2024-05-14T17:00:10.065755+0000","last_active":"2024-05-14T17:03:37.789250+0000","last_peered":"2024-05-14T17:03:37.789250+0000","last_clean":"2024-05-14T17:03:37.789250+0000","last_became_active":"2024-05-14T17:00:10.065390+0000","last_became_peered":"2024-05-14T17:00:10.065390+0000","last_unstale":"2024-05-14T17:03:37.789250+0000","last_undegraded":"2024-05-14T17:03:37.789250+0000","last_fullsized":"2024-05-14T17:03:37.789250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:24.632410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688103+0000","last_change":"2024-05-14T17:00:07.039176+0000","last_active":"2024-05-14T17:03:37.688103+0000","last_peered":"2024-05-14T17:03:37.688103+0000","last_clean":"2024-05-14T17:03:37.688103+0000","last_became_active":"2024-05-14T17:00:07.038746+0000","last_became_peered":"2024-05-14T17:00:07.038746+0000","last_unstale":"2024-05-14T17:03:37.688103+0000","last_undegraded":"2024-05-14T17:03:37.688103+0000","last_fullsized":"2024-05-14T17:03:37.688103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:06.011764+0000","last_clean_scrub_stamp":"2024-05-14T17:00:06.011764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:19:54.525755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.789479+0000","last_change":"2024-05-14T17:00:10.060298+0000","last_active":"2024-05-14T17:03:37.789479+0000","last_peered":"2024-05-14T17:03:37.789479+0000","last_clean":"2024-05-14T17:03:37.789479+0000","last_became_active":"2024-05-14T17:00:10.059251+0000","last_became_peered":"2024-05-14T17:00:10.059251+0000","last_unstale":"2024-05-14T17:03:37.789479+0000","last_undegraded":"2024-05-14T17:03:37.789479+0000","last_fullsized":"2024-05-14T17:03:37.789479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:41:25.933607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687096+0000","last_change":"2024-05-14T17:00:10.067183+0000","last_active":"2024-05-14T17:03:37.687096+0000","last_peered":"2024-05-14T17:03:37.687096+0000","last_clean":"2024-05-14T17:03:37.687096+0000","last_became_active":"2024-05-14T17:00:10.066084+0000","last_became_peered":"2024-05-14T17:00:10.066084+0000","last_unstale":"2024-05-14T17:03:37.687096+0000","last_undegraded":"2024-05-14T17:03:37.687096+0000","last_fullsized":"2024-05-14T17:03:37.687096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:43:46.986896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575519+0000","last_change":"2024-05-14T17:00:10.066621+0000","last_active":"2024-05-14T17:03:37.575519+0000","last_peered":"2024-05-14T17:03:37.575519+0000","last_clean":"2024-05-14T17:03:37.575519+0000","last_became_active":"2024-05-14T17:00:10.066280+0000","last_became_peered":"2024-05-14T17:00:10.066280+0000","last_unstale":"2024-05-14T17:03:37.575519+0000","last_undegraded":"2024-05-14T17:03:37.575519+0000","last_fullsized":"2024-05-14T17:03:37.575519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:16:53.045242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575519+0000","last_change":"2024-05-14T17:00:10.064467+0000","last_active":"2024-05-14T17:03:37.575519+0000","last_peered":"2024-05-14T17:03:37.575519+0000","last_clean":"2024-05-14T17:03:37.575519+0000","last_became_active":"2024-05-14T17:00:10.064074+0000","last_became_peered":"2024-05-14T17:00:10.064074+0000","last_unstale":"2024-05-14T17:03:37.575519+0000","last_undegraded":"2024-05-14T17:03:37.575519+0000","last_fullsized":"2024-05-14T17:03:37.575519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:10:39.003369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687177+0000","last_change":"2024-05-14T17:00:10.056766+0000","last_active":"2024-05-14T17:03:37.687177+0000","last_peered":"2024-05-14T17:03:37.687177+0000","last_clean":"2024-05-14T17:03:37.687177+0000","last_became_active":"2024-05-14T17:00:10.055617+0000","last_became_peered":"2024-05-14T17:00:10.055617+0000","last_unstale":"2024-05-14T17:03:37.687177+0000","last_undegraded":"2024-05-14T17:03:37.687177+0000","last_fullsized":"2024-05-14T17:03:37.687177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:41:23.360747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688376+0000","last_change":"2024-05-14T17:00:10.059458+0000","last_active":"2024-05-14T17:03:37.688376+0000","last_peered":"2024-05-14T17:03:37.688376+0000","last_clean":"2024-05-14T17:03:37.688376+0000","last_became_active":"2024-05-14T17:00:10.056154+0000","last_became_peered":"2024-05-14T17:00:10.056154+0000","last_unstale":"2024-05-14T17:03:37.688376+0000","last_undegraded":"2024-05-14T17:03:37.688376+0000","last_fullsized":"2024-05-14T17:03:37.688376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:45.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687233+0000","last_change":"2024-05-14T17:00:10.062212+0000","last_active":"2024-05-14T17:03:37.687233+0000","last_peered":"2024-05-14T17:03:37.687233+0000","last_clean":"2024-05-14T17:03:37.687233+0000","last_became_active":"2024-05-14T17:00:10.057447+0000","last_became_peered":"2024-05-14T17:00:10.057447+0000","last_unstale":"2024-05-14T17:03:37.687233+0000","last_undegraded":"2024-05-14T17:03:37.687233+0000","last_fullsized":"2024-05-14T17:03:37.687233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:37.447291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.575880+0000","last_change":"2024-05-14T17:00:10.062281+0000","last_active":"2024-05-14T17:03:37.575880+0000","last_peered":"2024-05-14T17:03:37.575880+0000","last_clean":"2024-05-14T17:03:37.575880+0000","last_became_active":"2024-05-14T17:00:10.061916+0000","last_became_peered":"2024-05-14T17:00:10.061916+0000","last_unstale":"2024-05-14T17:03:37.575880+0000","last_undegraded":"2024-05-14T17:03:37.575880+0000","last_fullsized":"2024-05-14T17:03:37.575880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:12:06.483801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.674537+0000","last_change":"2024-05-14T17:01:04.238941+0000","last_active":"2024-05-14T17:03:37.674537+0000","last_peered":"2024-05-14T17:03:37.674537+0000","last_clean":"2024-05-14T17:03:37.674537+0000","last_became_active":"2024-05-14T17:01:04.238526+0000","last_became_peered":"2024-05-14T17:01:04.238526+0000","last_unstale":"2024-05-14T17:03:37.674537+0000","last_undegraded":"2024-05-14T17:03:37.674537+0000","last_fullsized":"2024-05-14T17:03:37.674537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:41:40.081665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.680279+0000","last_change":"2024-05-14T17:00:10.059531+0000","last_active":"2024-05-14T17:03:37.680279+0000","last_peered":"2024-05-14T17:03:37.680279+0000","last_clean":"2024-05-14T17:03:37.680279+0000","last_became_active":"2024-05-14T17:00:10.059208+0000","last_became_peered":"2024-05-14T17:00:10.059208+0000","last_unstale":"2024-05-14T17:03:37.680279+0000","last_undegraded":"2024-05-14T17:03:37.680279+0000","last_fullsized":"2024-05-14T17:03:37.680279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:54:24.049807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.673958+0000","last_change":"2024-05-14T17:00:10.070207+0000","last_active":"2024-05-14T17:03:37.673958+0000","last_peered":"2024-05-14T17:03:37.673958+0000","last_clean":"2024-05-14T17:03:37.673958+0000","last_became_active":"2024-05-14T17:00:10.069264+0000","last_became_peered":"2024-05-14T17:00:10.069264+0000","last_unstale":"2024-05-14T17:03:37.673958+0000","last_undegraded":"2024-05-14T17:03:37.673958+0000","last_fullsized":"2024-05-14T17:03:37.673958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:04:55.668315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.686877+0000","last_change":"2024-05-14T17:00:10.067858+0000","last_active":"2024-05-14T17:03:37.686877+0000","last_peered":"2024-05-14T17:03:37.686877+0000","last_clean":"2024-05-14T17:03:37.686877+0000","last_became_active":"2024-05-14T17:00:10.066278+0000","last_became_peered":"2024-05-14T17:00:10.066278+0000","last_unstale":"2024-05-14T17:03:37.686877+0000","last_undegraded":"2024-05-14T17:03:37.686877+0000","last_fullsized":"2024-05-14T17:03:37.686877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:11:07.971510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.688787+0000","last_change":"2024-05-14T17:00:10.064596+0000","last_active":"2024-05-14T17:03:37.688787+0000","last_peered":"2024-05-14T17:03:37.688787+0000","last_clean":"2024-05-14T17:03:37.688787+0000","last_became_active":"2024-05-14T17:00:10.063722+0000","last_became_peered":"2024-05-14T17:00:10.063722+0000","last_unstale":"2024-05-14T17:03:37.688787+0000","last_undegraded":"2024-05-14T17:03:37.688787+0000","last_fullsized":"2024-05-14T17:03:37.688787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:35:56.756716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-14T17:03:37.687085+0000","last_change":"2024-05-14T17:01:04.239386+0000","last_active":"2024-05-14T17:03:37.687085+0000","last_peered":"2024-05-14T17:03:37.687085+0000","last_clean":"2024-05-14T17:03:37.687085+0000","last_became_active":"2024-05-14T17:01:04.239053+0000","last_became_peered":"2024-05-14T17:01:04.239053+0000","last_unstale":"2024-05-14T17:03:37.687085+0000","last_undegraded":"2024-05-14T17:03:37.687085+0000","last_fullsized":"2024-05-14T17:03:37.687085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T17:00:09.031000+0000","last_clean_scrub_stamp":"2024-05-14T17:00:09.031000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:42:49.304566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793822,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1642,"num_read_kb":1034,"num_write":417,"num_write_kb":2351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1047,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2381466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645374,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":441660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841602,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29028,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716124,"statfs":{"total":95995035648,"available":95965310976,"internally_reserved":0,"allocated":1937408,"data_stored":1428789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005127,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1310720,"data_stored":835078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201357,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36616,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708536,"statfs":{"total":95995035648,"available":95957540864,"internally_reserved":0,"allocated":1310720,"data_stored":834998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397584,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1035026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561110,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":905216,"data_stored":441251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":508103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":442522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":508797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149797,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1163010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953567,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757340,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37268,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707884,"statfs":{"total":95995035648,"available":95956873216,"internally_reserved":0,"allocated":1855488,"data_stored":1362551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T17:04:02.673 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:02.927 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:02.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:02 smithi150 ceph-mon[36204]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-14T17:04:02.967 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:02 smithi150 ceph-mon[36204]: mgrmap e25: x(active, since 24s), standbys: y, z 2024-05-14T17:04:02.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:02 smithi150 ceph-mon[36204]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:04:03.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:02 smithi099 ceph-mon[31851]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-14T17:04:03.030 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:02 smithi099 ceph-mon[31851]: mgrmap e25: x(active, since 24s), standbys: y, z 2024-05-14T17:04:03.045 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:02 smithi099 ceph-mon[31851]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:04:03.059 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:02 smithi193 ceph-mon[35962]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-05-14T17:04:03.072 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:02 smithi193 ceph-mon[35962]: mgrmap e25: x(active, since 24s), standbys: y, z 2024-05-14T17:04:03.131 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:02 smithi193 ceph-mon[35962]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T17:04:04.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 -- 172.21.15.99:0/2690748114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006b220 msgr2=0x7f61c006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:04.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 --2- 172.21.15.99:0/2690748114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006b220 0x7f61c006b6a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009990 tx=0x7f61b4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 -- 172.21.15.99:0/2690748114 shutdown_connections 2024-05-14T17:04:04.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 --2- 172.21.15.99:0/2690748114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c006bbe0 0x7f61c01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 --2- 172.21.15.99:0/2690748114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c01095b0 0x7f61c0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 --2- 172.21.15.99:0/2690748114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006b220 0x7f61c006b6a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:04.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 -- 172.21.15.99:0/2690748114 >> 172.21.15.99:0/2690748114 conn(0x7f61c00793f0 msgr2=0x7f61c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:04.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 -- 172.21.15.99:0/2690748114 shutdown_connections 2024-05-14T17:04:04.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.653+0000 7f61c527e700 1 -- 172.21.15.99:0/2690748114 wait complete. 2024-05-14T17:04:04.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.654+0000 7f61c527e700 1 Processor -- start 2024-05-14T17:04:04.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.654+0000 7f61c527e700 1 -- start start 2024-05-14T17:04:04.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.654+0000 7f61c527e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c006b220 0x7f61c019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.654+0000 7f61c527e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 0x7f61c019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.654+0000 7f61c527e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c01095b0 0x7f61c0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:04.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61c527e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0113760 con 0x7f61c006bbe0 2024-05-14T17:04:04.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61c527e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01135e0 con 0x7f61c006b220 2024-05-14T17:04:04.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61c527e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61c01138e0 con 0x7f61c01095b0 2024-05-14T17:04:04.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 0x7f61c019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 0x7f61c019ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35186/0 (socket says 172.21.15.99:35186) 2024-05-14T17:04:05.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 -- 172.21.15.99:0/874766654 learned_addr learned my addr 172.21.15.99:0/874766654 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61beffd700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c006b220 0x7f61c019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61bf7fe700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c01095b0 0x7f61c0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 -- 172.21.15.99:0/874766654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c01095b0 msgr2=0x7f61c0199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:05.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c01095b0 0x7f61c0199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 -- 172.21.15.99:0/874766654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c006b220 msgr2=0x7f61c019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:05.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c006b220 0x7f61c019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 -- 172.21.15.99:0/874766654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c019a400 con 0x7f61c006bbe0 2024-05-14T17:04:05.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61beffd700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c006b220 0x7f61c019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:05.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61be7fc700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 0x7f61c019ff90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000c00 tx=0x7f61b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:05.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4027070 con 0x7f61c006bbe0 2024-05-14T17:04:05.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c019a690 con 0x7f61c006bbe0 2024-05-14T17:04:05.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b4003fa0 con 0x7f61c006bbe0 2024-05-14T17:04:05.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4022db0 con 0x7f61c006bbe0 2024-05-14T17:04:05.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.655+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a6800 con 0x7f61c006bbe0 2024-05-14T17:04:05.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.656+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c010ba90 con 0x7f61c006bbe0 2024-05-14T17:04:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.658+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f61b402a070 con 0x7f61c006bbe0 2024-05-14T17:04:05.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.658+0000 7f61a7fff700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f61a80b2b10 0x7f61a80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.658+0000 7f61beffd700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f61a80b2b10 0x7f61a80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.658+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f61b40e3960 con 0x7f61c006bbe0 2024-05-14T17:04:05.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.659+0000 7f61beffd700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f61a80b2b10 0x7f61a80b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f61b00099d0 tx=0x7f61b0008040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:05.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.662+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61b4013070 con 0x7f61c006bbe0 2024-05-14T17:04:05.402 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:04 smithi193 ceph-mon[35962]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 53 op/s 2024-05-14T17:04:05.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:04 smithi150 ceph-mon[36204]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 53 op/s 2024-05-14T17:04:05.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.918+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61c019afe0 con 0x7f61c006bbe0 2024-05-14T17:04:05.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:04 smithi099 ceph-mon[31851]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 53 op/s 2024-05-14T17:04:05.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.919+0000 7f61a7fff700 1 -- 172.21.15.99:0/874766654 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f61b40ab1b0 con 0x7f61c006bbe0 2024-05-14T17:04:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:04:05.459 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:04:05.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f61a80b2b10 msgr2=0x7f61a80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:05.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f61a80b2b10 0x7f61a80b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f61b00099d0 tx=0x7f61b0008040 comp rx=0 tx=0).stop 2024-05-14T17:04:05.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 msgr2=0x7f61c019ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:05.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 0x7f61c019ff90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000c00 tx=0x7f61b40047b0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 shutdown_connections 2024-05-14T17:04:05.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f61a80b2b10 0x7f61a80b4fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61c01095b0 0x7f61c0199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c006b220 0x7f61c019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 --2- 172.21.15.99:0/874766654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c006bbe0 0x7f61c019ff90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.921+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 >> 172.21.15.99:0/874766654 conn(0x7f61c00793f0 msgr2=0x7f61c01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:05.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.922+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 shutdown_connections 2024-05-14T17:04:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:04.922+0000 7f61c527e700 1 -- 172.21.15.99:0/874766654 wait complete. 2024-05-14T17:04:05.659 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:04:05.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:05.709 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:05 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/874766654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:05.935 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:05.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:05 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/874766654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:06.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:05 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/874766654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:06.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:06 smithi150 ceph-mon[36204]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-05-14T17:04:07.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:06 smithi099 ceph-mon[31851]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-05-14T17:04:07.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:06 smithi193 ceph-mon[35962]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-05-14T17:04:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 -- 172.21.15.99:0/2753933659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 msgr2=0x7f3f1010e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 --2- 172.21.15.99:0/2753933659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f1010e7c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009ca0 tx=0x7f3f0802a8c0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 -- 172.21.15.99:0/2753933659 shutdown_connections 2024-05-14T17:04:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 --2- 172.21.15.99:0/2753933659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f1010e7c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 --2- 172.21.15.99:0/2753933659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10101ba0 0x7f3f10102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 --2- 172.21.15.99:0/2753933659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f10107970 0x7f3f10107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.227+0000 7f3f1775f640 1 -- 172.21.15.99:0/2753933659 >> 172.21.15.99:0/2753933659 conn(0x7f3f100787c0 msgr2=0x7f3f10078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 -- 172.21.15.99:0/2753933659 shutdown_connections 2024-05-14T17:04:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 -- 172.21.15.99:0/2753933659 wait complete. 2024-05-14T17:04:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 Processor -- start 2024-05-14T17:04:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 -- start start 2024-05-14T17:04:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f10101ba0 0x7f3f10198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f10198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10107970 0x7f3f1019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10111c10 con 0x7f3f10102540 2024-05-14T17:04:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10111a90 con 0x7f3f10101ba0 2024-05-14T17:04:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f1775f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f10111d90 con 0x7f3f10107970 2024-05-14T17:04:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f14cfa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f10198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f14cfa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f10198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35198/0 (socket says 172.21.15.99:35198) 2024-05-14T17:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.228+0000 7f3f14cfa640 1 -- 172.21.15.99:0/335841856 learned_addr learned my addr 172.21.15.99:0/335841856 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f154fb640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f10101ba0 0x7f3f10198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10107970 0x7f3f1019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f14cfa640 1 -- 172.21.15.99:0/335841856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10107970 msgr2=0x7f3f1019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10107970 0x7f3f1019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f14cfa640 1 -- 172.21.15.99:0/335841856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f10101ba0 msgr2=0x7f3f10198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f10101ba0 0x7f3f10198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f14cfa640 1 -- 172.21.15.99:0/335841856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1019d000 con 0x7f3f10102540 2024-05-14T17:04:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f10198550 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000d770 tx=0x7f3f0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f067fc640 1 -- 172.21.15.99:0/335841856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00017070 con 0x7f3f10102540 2024-05-14T17:04:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1019d2f0 con 0x7f3f10102540 2024-05-14T17:04:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f067fc640 1 -- 172.21.15.99:0/335841856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f000054d0 con 0x7f3f10102540 2024-05-14T17:04:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f067fc640 1 -- 172.21.15.99:0/335841856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00004c30 con 0x7f3f10102540 2024-05-14T17:04:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.229+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a4b80 con 0x7f3f10102540 2024-05-14T17:04:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.231+0000 7f3f067fc640 1 -- 172.21.15.99:0/335841856 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f3f00004e10 con 0x7f3f10102540 2024-05-14T17:04:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 Processor -- start 2024-05-14T17:04:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 -- start start 2024-05-14T17:04:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x56438382bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383836590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x564383838ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56438379ec30 con 0x564383836ad0 2024-05-14T17:04:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56438378e7d0 con 0x56438382b8f0 2024-05-14T17:04:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564383792810 con 0x56438382c210 2024-05-14T17:04:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x564383838ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x564383838ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35204/0 (socket says 172.21.15.99:35204) 2024-05-14T17:04:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 -- 172.21.15.99:0/2380441127 learned_addr learned my addr 172.21.15.99:0/2380441127 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f154fb640 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383836590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f067fc640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3efc0b2b30 0x7f3efc0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x56438382bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3efc0b2b30 0x7f3efc0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 -- 172.21.15.99:0/2380441127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 msgr2=0x564383836590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383836590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 -- 172.21.15.99:0/2380441127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 msgr2=0x56438382bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.232+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x56438382bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3f067fc640 1 -- 172.21.15.99:0/335841856 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3f000e2460 con 0x7f3f10102540 2024-05-14T17:04:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3f14cfa640 1 -- 172.21.15.99:0/2380441127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643834a3210 con 0x564383836ad0 2024-05-14T17:04:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x56438382bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x564383838ec0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3f000dfaf0 tx=0x7f3f00009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e8e100627609c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3efb7fe640 1 -- 172.21.15.99:0/2380441127 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00020020 con 0x564383836ad0 2024-05-14T17:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3efc0b2b30 0x7f3efc0b4ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004af0 tx=0x7f3f080095c0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3efb7fe640 1 -- 172.21.15.99:0/2380441127 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f000a9030 con 0x564383836ad0 2024-05-14T17:04:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.233+0000 7f3efb7fe640 1 -- 172.21.15.99:0/2380441127 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f000a9520 con 0x564383836ad0 2024-05-14T17:04:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2380441127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 msgr2=0x564383838ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x564383838ec0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3f000dfaf0 tx=0x7f3f00009f90 comp rx=0 tx=0).stop 2024-05-14T17:04:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2380441127 shutdown_connections 2024-05-14T17:04:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x564383838ec0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383836590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2380441127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x56438382bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2380441127 >> 172.21.15.99:0/2380441127 conn(0x564383827e00 msgr2=0x5643838281e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2380441127 shutdown_connections 2024-05-14T17:04:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2380441127 wait complete. 2024-05-14T17:04:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 -- start start 2024-05-14T17:04:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.234+0000 7f3f1a47a740 1 Processor -- start 2024-05-14T17:04:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x564383836470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383833760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564383836ad0 0x564383833d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564383474790 con 0x56438382b8f0 2024-05-14T17:04:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56438374a890 con 0x564383836ad0 2024-05-14T17:04:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5643838396e0 con 0x56438382c210 2024-05-14T17:04:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 10 client.0 _ll_register_callbacks cb 0x56438374dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x564383836470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x564383836470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35208/0 (socket says 172.21.15.99:35208) 2024-05-14T17:04:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 -- 172.21.15.99:0/2624079774 learned_addr learned my addr 172.21.15.99:0/2624079774 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383833760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564383836ad0 0x564383833d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 -- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 msgr2=0x564383833760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383833760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 -- 172.21.15.99:0/2624079774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564383836ad0 msgr2=0x564383833d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564383836ad0 0x564383833d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 -- 172.21.15.99:0/2624079774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438356f620 con 0x56438382b8f0 2024-05-14T17:04:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564383836ad0 0x564383833d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f154fb640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x564383836470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c012700 tx=0x7f3f0c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dad27afef8c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3ef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/2624079774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c004240 con 0x56438382b8f0 2024-05-14T17:04:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/2624079774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3f0c0043e0 con 0x56438382b8f0 2024-05-14T17:04:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f000a9030 con 0x56438382b8f0 2024-05-14T17:04:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/2624079774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c0119a0 con 0x56438382b8f0 2024-05-14T17:04:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 10 client.15282 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.235+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643838451f0 con 0x56438382b8f0 2024-05-14T17:04:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/2624079774 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f000a9030 con 0x56438382b8f0 2024-05-14T17:04:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/2624079774 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3f0c00f460 con 0x56438382b8f0 2024-05-14T17:04:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f1a47a740 10 client.15282 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f1a47a740 10 client.15282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f1a47a740 20 client.15282 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564383846700 0x564383848af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f1a47a740 4 client.15282 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643837a0b80 con 0x564383846700 2024-05-14T17:04:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.236+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564383846700 0x564383848af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564383846700 0x564383848af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c770dd186b1552 server_cookie=de46cc9efb2ff773 in_seq=0 out_seq=0 2024-05-14T17:04:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3ef8ff9640 10 client.15282 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/2624079774 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5643837a0b80 con 0x564383846700 2024-05-14T17:04:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3ef8ff9640 10 client.15282 handle_command_reply: tid=0 2024-05-14T17:04:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 client.15282 shutdown 2024-05-14T17:04:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 msgr2=0x564383836470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x564383836470 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c012700 tx=0x7f3f0c012bc0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 shutdown_connections 2024-05-14T17:04:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564383846700 0x564383848af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564383836ad0 0x564383833d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x564383833760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.237+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/2624079774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x564383836470 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.238+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 >> 172.21.15.99:0/2624079774 conn(0x564383827e00 msgr2=0x564383827100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.238+0000 7f3ef8ff9640 0 client.15282 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.238+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 shutdown_connections 2024-05-14T17:04:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.238+0000 7f3f1a47a740 1 -- 172.21.15.99:0/2624079774 wait complete. 2024-05-14T17:04:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.238+0000 7f3f1a47a740 20 client.15282 trim_cache size 0 max 16384 2024-05-14T17:04:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 Processor -- start 2024-05-14T17:04:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 -- start start 2024-05-14T17:04:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x5643838425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x56438384e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x56438384e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101afcd0 con 0x564383836ad0 2024-05-14T17:04:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ad5a0 con 0x56438382b8f0 2024-05-14T17:04:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ab3f0 con 0x56438382c210 2024-05-14T17:04:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x5643838425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x5643838425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:37032/0 (socket says 172.21.15.99:37032) 2024-05-14T17:04:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 -- 172.21.15.99:0/3934026865 learned_addr learned my addr 172.21.15.99:0/3934026865 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f14cfa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x56438384e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 -- 172.21.15.99:0/3934026865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 msgr2=0x56438384e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x56438384e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 -- 172.21.15.99:0/3934026865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 msgr2=0x56438384e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x56438384e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 -- 172.21.15.99:0/3934026865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564383828780 con 0x56438382b8f0 2024-05-14T17:04:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.253+0000 7f3f154fb640 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x5643838425b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00f8d0 tx=0x7f3f0c0058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ff2e16645c33a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3efaffd640 1 -- 172.21.15.99:0/3934026865 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c029040 con 0x56438382b8f0 2024-05-14T17:04:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3efaffd640 1 -- 172.21.15.99:0/3934026865 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c00fe90 con 0x56438382b8f0 2024-05-14T17:04:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3efaffd640 1 -- 172.21.15.99:0/3934026865 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c003850 con 0x56438382b8f0 2024-05-14T17:04:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3934026865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 msgr2=0x5643838425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x5643838425b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00f8d0 tx=0x7f3f0c0058b0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3934026865 shutdown_connections 2024-05-14T17:04:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564383836ad0 0x56438384e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56438382c210 0x56438384e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3934026865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382b8f0 0x5643838425b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3934026865 >> 172.21.15.99:0/3934026865 conn(0x564383827e00 msgr2=0x564383827100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3934026865 shutdown_connections 2024-05-14T17:04:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.254+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3934026865 wait complete. 2024-05-14T17:04:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 -- start start 2024-05-14T17:04:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 Processor -- start 2024-05-14T17:04:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x56438383c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382c210 0x56438383cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564383836ad0 0x56438383d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101aba60 con 0x56438382b8f0 2024-05-14T17:04:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101aced0 con 0x56438382c210 2024-05-14T17:04:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f101ac130 con 0x564383836ad0 2024-05-14T17:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f1a47a740 10 client.0 _ll_register_callbacks cb 0x7f3f0c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x56438383c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x56438383c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35240/0 (socket says 172.21.15.99:35240) 2024-05-14T17:04:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 -- 172.21.15.99:0/3260227337 learned_addr learned my addr 172.21.15.99:0/3260227337 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382c210 0x56438383cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f15cfc640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564383836ad0 0x56438383d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 -- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564383836ad0 msgr2=0x56438383d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564383836ad0 0x56438383d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 -- 172.21.15.99:0/3260227337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382c210 msgr2=0x56438383cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382c210 0x56438383cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.255+0000 7f3f154fb640 1 -- 172.21.15.99:0/3260227337 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643838b0070 con 0x56438382b8f0 2024-05-14T17:04:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f154fb640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x56438383c5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c005fa0 tx=0x7f3f0c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc72610900ffb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3ef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/3260227337 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c029070 con 0x56438382b8f0 2024-05-14T17:04:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/3260227337 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3f0c00fa40 con 0x56438382b8f0 2024-05-14T17:04:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c00fe90 con 0x56438382b8f0 2024-05-14T17:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/3260227337 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c00fc40 con 0x56438382b8f0 2024-05-14T17:04:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643838b1380 con 0x56438382b8f0 2024-05-14T17:04:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/3260227337 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c00fe90 con 0x56438382b8f0 2024-05-14T17:04:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/3260227337 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3f0c02c070 con 0x56438382b8f0 2024-05-14T17:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 10 client.15285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 20 client.15285 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5643838b1990 0x5643838b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 4 client.15285 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.256+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56438379a360 con 0x5643838b1990 2024-05-14T17:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.257+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5643838b1990 0x5643838b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.257+0000 7f3f14cfa640 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5643838b1990 0x5643838b3db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d57dbb25106372 server_cookie=2e0d19c403f97131 in_seq=0 out_seq=0 2024-05-14T17:04:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.257+0000 7f3ef8ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.269+0000 7f3ef8ff9640 1 -- 172.21.15.99:0/3260227337 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56438379a360 con 0x5643838b1990 2024-05-14T17:04:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.269+0000 7f3ef8ff9640 10 client.15285 handle_command_reply: tid=0 2024-05-14T17:04:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.269+0000 7f3f1a47a740 1 client.15285 shutdown 2024-05-14T17:04:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.269+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 msgr2=0x56438383c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.269+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x56438383c5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c005fa0 tx=0x7f3f0c00f460 comp rx=0 tx=0).stop 2024-05-14T17:04:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 shutdown_connections 2024-05-14T17:04:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5643838b1990 0x5643838b3db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564383836ad0 0x56438383d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56438382c210 0x56438383cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3ef8ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 --2- 172.21.15.99:0/3260227337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438382b8f0 0x56438383c5e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 >> 172.21.15.99:0/3260227337 conn(0x564383827e00 msgr2=0x5643838415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 shutdown_connections 2024-05-14T17:04:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 1 -- 172.21.15.99:0/3260227337 wait complete. 2024-05-14T17:04:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.270+0000 7f3f1a47a740 20 client.15285 trim_cache size 0 max 16384 2024-05-14T17:04:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:04:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:04:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:04:10.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:08 smithi193 ceph-mon[35962]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-14T17:04:10.548 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:08 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3670236845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:10.558 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:08 smithi150 ceph-mon[36204]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-14T17:04:10.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:08 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3670236845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3efc0b2b30 msgr2=0x7f3efc0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3efc0b2b30 0x7f3efc0b4ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004af0 tx=0x7f3f080095c0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 msgr2=0x7f3f10198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f10198550 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000d770 tx=0x7f3f0000dc30 comp rx=0 tx=0).stop 2024-05-14T17:04:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 shutdown_connections 2024-05-14T17:04:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3efc0b2b30 0x7f3efc0b4ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f10107970 0x7f3f1019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f10102540 0x7f3f10198550 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 --2- 172.21.15.99:0/335841856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f10101ba0 0x7f3f10198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 >> 172.21.15.99:0/335841856 conn(0x7f3f100787c0 msgr2=0x7f3f10100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 shutdown_connections 2024-05-14T17:04:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:07.271+0000 7f3f1775f640 1 -- 172.21.15.99:0/335841856 wait complete. 2024-05-14T17:04:10.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 -- 172.21.15.99:0/537079817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 msgr2=0x7f3bb01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/537079817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb01028c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009990 tx=0x7f3b9c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 -- 172.21.15.99:0/537079817 shutdown_connections 2024-05-14T17:04:10.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/537079817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 0x7f3bb010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/537079817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 0x7f3bb0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/537079817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb01028c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 -- 172.21.15.99:0/537079817 >> 172.21.15.99:0/537079817 conn(0x7f3bb00793f0 msgr2=0x7f3bb0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:10.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 -- 172.21.15.99:0/537079817 shutdown_connections 2024-05-14T17:04:10.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.548+0000 7f3bb6b95700 1 -- 172.21.15.99:0/537079817 wait complete. 2024-05-14T17:04:10.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 Processor -- start 2024-05-14T17:04:10.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 -- start start 2024-05-14T17:04:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 0x7f3bb019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 0x7f3bb0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01125f0 con 0x7f3bb0102440 2024-05-14T17:04:10.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0112470 con 0x7f3bb0102e00 2024-05-14T17:04:10.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb6b95700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0112770 con 0x7f3bb0108520 2024-05-14T17:04:10.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb4931700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb4931700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35264/0 (socket says 172.21.15.99:35264) 2024-05-14T17:04:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3bb4931700 1 -- 172.21.15.99:0/3670236845 learned_addr learned my addr 172.21.15.99:0/3670236845 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:10.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.549+0000 7f3baffff700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 0x7f3bb019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb5132700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 0x7f3bb0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb4931700 1 -- 172.21.15.99:0/3670236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 msgr2=0x7f3bb0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb4931700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 0x7f3bb0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb4931700 1 -- 172.21.15.99:0/3670236845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 msgr2=0x7f3bb019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb4931700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 0x7f3bb019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb4931700 1 -- 172.21.15.99:0/3670236845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb019a260 con 0x7f3bb0102440 2024-05-14T17:04:11.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb5132700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 0x7f3bb0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:11.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3baffff700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 0x7f3bb019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:11.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb4931700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb019f940 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400ef30 tx=0x7f3ba400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:11.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40092e0 con 0x7f3bb0102440 2024-05-14T17:04:11.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba4004bb0 con 0x7f3bb0102440 2024-05-14T17:04:11.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb019a4f0 con 0x7f3bb0102440 2024-05-14T17:04:11.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004280 con 0x7f3bb0102440 2024-05-14T17:04:11.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.550+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a6810 con 0x7f3bb0102440 2024-05-14T17:04:11.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.551+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb006a230 con 0x7f3bb0102440 2024-05-14T17:04:11.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.552+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f3ba401b030 con 0x7f3bb0102440 2024-05-14T17:04:11.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.553+0000 7f3badffb700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3ba00b2b10 0x7f3ba00b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.553+0000 7f3baffff700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3ba00b2b10 0x7f3ba00b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:11.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.553+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3ba40e2560 con 0x7f3bb0102440 2024-05-14T17:04:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.553+0000 7f3baffff700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3ba00b2b10 0x7f3ba00b4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b2e0 tx=0x7f3b9c01a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.558+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ba4022020 con 0x7f3bb0102440 2024-05-14T17:04:11.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.804+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb002e560 con 0x7f3bb0102440 2024-05-14T17:04:11.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.805+0000 7f3badffb700 1 -- 172.21.15.99:0/3670236845 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f3ba4010040 con 0x7f3bb0102440 2024-05-14T17:04:11.176 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:04:11.219 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:04:11.228 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:10 smithi150 ceph-mon[36204]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-05-14T17:04:11.236 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:10 smithi193 ceph-mon[35962]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-05-14T17:04:11.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3ba00b2b10 msgr2=0x7f3ba00b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:11.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3ba00b2b10 0x7f3ba00b4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b2e0 tx=0x7f3b9c01a040 comp rx=0 tx=0).stop 2024-05-14T17:04:11.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 msgr2=0x7f3bb019f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:11.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb019f940 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400ef30 tx=0x7f3ba400c450 comp rx=0 tx=0).stop 2024-05-14T17:04:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 shutdown_connections 2024-05-14T17:04:11.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3ba00b2b10 0x7f3ba00b4fd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bb0108520 0x7f3bb0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb0102440 0x7f3bb019f940 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 --2- 172.21.15.99:0/3670236845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb0102e00 0x7f3bb019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 >> 172.21.15.99:0/3670236845 conn(0x7f3bb00793f0 msgr2=0x7f3bb006b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:11.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 shutdown_connections 2024-05-14T17:04:11.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:07.807+0000 7f3bb6b95700 1 -- 172.21.15.99:0/3670236845 wait complete. 2024-05-14T17:04:11.357 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:04:11.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:04:11.438 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:08 smithi099 ceph-mon[31851]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-14T17:04:11.448 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:08 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3670236845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:11.457 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:10 smithi099 ceph-mon[31851]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-05-14T17:04:11.722 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:12 smithi193 ceph-mon[35962]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-05-14T17:04:13.050 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:12 smithi099 ceph-mon[31851]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-05-14T17:04:13.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:12 smithi150 ceph-mon[36204]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-05-14T17:04:13.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 -- 172.21.15.99:0/863074303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102e00 msgr2=0x7f441410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 --2- 172.21.15.99:0/863074303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102e00 0x7f441410f2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f440c00b0a0 tx=0x7f440c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 -- 172.21.15.99:0/863074303 shutdown_connections 2024-05-14T17:04:13.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 --2- 172.21.15.99:0/863074303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102e00 0x7f441410f2f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 --2- 172.21.15.99:0/863074303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414108520 0x7f4414108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 --2- 172.21.15.99:0/863074303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4414102440 0x7f44141028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 -- 172.21.15.99:0/863074303 >> 172.21.15.99:0/863074303 conn(0x7f44140793f0 msgr2=0x7f4414079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:13.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 -- 172.21.15.99:0/863074303 shutdown_connections 2024-05-14T17:04:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.418+0000 7f441bbce700 1 -- 172.21.15.99:0/863074303 wait complete. 2024-05-14T17:04:13.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 Processor -- start 2024-05-14T17:04:13.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 -- start start 2024-05-14T17:04:13.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 0x7f441419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414102e00 0x7f441419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4414108520 0x7f44141a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44141125f0 con 0x7f4414102440 2024-05-14T17:04:13.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112470 con 0x7f4414102e00 2024-05-14T17:04:13.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 0x7f441419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.419+0000 7f441bbce700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112770 con 0x7f4414108520 2024-05-14T17:04:13.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 0x7f441419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35270/0 (socket says 172.21.15.99:35270) 2024-05-14T17:04:13.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 -- 172.21.15.99:0/3140671000 learned_addr learned my addr 172.21.15.99:0/3140671000 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:13.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441a16b700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4414108520 0x7f44141a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f4419169700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414102e00 0x7f441419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 -- 172.21.15.99:0/3140671000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4414108520 msgr2=0x7f44141a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4414108520 0x7f44141a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 -- 172.21.15.99:0/3140671000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414102e00 msgr2=0x7f441419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414102e00 0x7f441419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 -- 172.21.15.99:0/3140671000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44141a5fb0 con 0x7f4414102440 2024-05-14T17:04:13.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f4419169700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414102e00 0x7f441419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:13.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441996a700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 0x7f441419a9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f440400c9b0 tx=0x7f440400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:13.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404007b00 con 0x7f4414102440 2024-05-14T17:04:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404007ca0 con 0x7f4414102440 2024-05-14T17:04:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141a61e0 con 0x7f4414102440 2024-05-14T17:04:13.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404018660 con 0x7f4414102440 2024-05-14T17:04:13.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.420+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a6770 con 0x7f4414102440 2024-05-14T17:04:13.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.421+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441406a230 con 0x7f4414102440 2024-05-14T17:04:13.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.423+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f440400f450 con 0x7f4414102440 2024-05-14T17:04:13.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.424+0000 7f440affd700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f44000b2b90 0x7f44000b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.424+0000 7f4419169700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f44000b2b90 0x7f44000b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.424+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f44040e2930 con 0x7f4414102440 2024-05-14T17:04:13.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.425+0000 7f4419169700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f44000b2b90 0x7f44000b5050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4410005fd0 tx=0x7f4410005ee0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:13.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.428+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4404014030 con 0x7f4414102440 2024-05-14T17:04:13.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.632+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f441402e560 con 0x7f4414102440 2024-05-14T17:04:13.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.634+0000 7f440affd700 1 -- 172.21.15.99:0/3140671000 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23071 (secure 0 0 0) 0x7f44040aa170 con 0x7f4414102440 2024-05-14T17:04:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:04:14.026 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:03:37.568534+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/2498052825":"2024-05-15T17:03:37.568496+0000","172.21.15.99:6800/2498052825":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/1935223933":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/781830097":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/701736301":"2024-05-15T17:03:37.568496+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2842786850":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:04:14.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f44000b2b90 msgr2=0x7f44000b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f44000b2b90 0x7f44000b5050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4410005fd0 tx=0x7f4410005ee0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 msgr2=0x7f441419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:14.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 0x7f441419a9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f440400c9b0 tx=0x7f440400ccc0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 shutdown_connections 2024-05-14T17:04:14.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f44000b2b90 0x7f44000b5050 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4414108520 0x7f44141a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414102440 0x7f441419a9f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 --2- 172.21.15.99:0/3140671000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4414102e00 0x7f441419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 >> 172.21.15.99:0/3140671000 conn(0x7f44140793f0 msgr2=0x7f4414100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:14.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 shutdown_connections 2024-05-14T17:04:14.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:13.636+0000 7f441bbce700 1 -- 172.21.15.99:0/3140671000 wait complete. 2024-05-14T17:04:14.197 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:13 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3140671000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:04:14.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:13 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3140671000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:04:14.212 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:13 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3140671000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:04:14.293 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-14T17:04:14.543 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:14.935 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:14 smithi099 ceph-mon[31851]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-05-14T17:04:15.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:14 smithi193 ceph-mon[35962]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-05-14T17:04:15.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:14 smithi150 ceph-mon[36204]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-05-14T17:04:16.655 INFO:teuthology.orchestra.run.smithi099.stdout:{"run_dir":"/var/run/ceph"} 2024-05-14T17:04:16.710 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-14T17:04:16.717 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 710 2024-05-14T17:04:16.734 DEBUG:tasks.quiescer:Yielding 2024-05-14T17:04:16.744 INFO:teuthology.run_tasks:Running task workunit... 2024-05-14T17:04:16.766 INFO:tasks.workunit:Pulling workunits from ref 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T17:04:16.778 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-14T17:04:16.784 INFO:tasks.workunit:timeout=3h 2024-05-14T17:04:16.794 INFO:tasks.workunit:cleanup=True 2024-05-14T17:04:16.803 DEBUG:teuthology.orchestra.run.smithi099:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-14T17:04:16.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:16.857 INFO:teuthology.orchestra.run.smithi099.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-14T17:04:16.867 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-14T17:04:16.951 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-14T17:04:16.959 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T17:04:16.968 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-14T17:04:16.975 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-14 17:00:42.649691420 +0000 2024-05-14T17:04:16.982 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-14 17:00:42.649691420 +0000 2024-05-14T17:04:17.062 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-14 17:03:37.153934072 +0000 2024-05-14T17:04:17.069 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-14 17:00:42.649691420 +0000 2024-05-14T17:04:17.076 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-14T17:04:17.083 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-14T17:04:17.091 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:16 smithi099 ceph-mon[31851]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-14T17:04:17.100 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:17.142 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T17:04:17.198 INFO:tasks.workunit.client.0.smithi099.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-14T17:04:17.207 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:16 smithi150 ceph-mon[36204]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-14T17:04:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:16 smithi193 ceph-mon[35962]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-14T17:04:18.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 -- 172.21.15.99:0/531323470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41041a0 msgr2=0x7f05b41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:18.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/531323470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41041a0 0x7f05b41081c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f05a40099c0 tx=0x7f05a4009cd0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 -- 172.21.15.99:0/531323470 shutdown_connections 2024-05-14T17:04:18.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/531323470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41088f0 0x7f05b4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/531323470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41037f0 0x7f05b4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/531323470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41041a0 0x7f05b41081c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 -- 172.21.15.99:0/531323470 >> 172.21.15.99:0/531323470 conn(0x7f05b407a6a0 msgr2=0x7f05b407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 -- 172.21.15.99:0/531323470 shutdown_connections 2024-05-14T17:04:18.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 -- 172.21.15.99:0/531323470 wait complete. 2024-05-14T17:04:18.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 Processor -- start 2024-05-14T17:04:18.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.695+0000 7f05bb3f9700 1 -- start start 2024-05-14T17:04:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05bb3f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41037f0 0x7f05b4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:18.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05bb3f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41041a0 0x7f05b4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:18.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05bb3f9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 0x7f05b40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:18.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05bb3f9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41136b0 con 0x7f05b41088f0 2024-05-14T17:04:18.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05bb3f9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4113530 con 0x7f05b41041a0 2024-05-14T17:04:18.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05bb3f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4113830 con 0x7f05b41037f0 2024-05-14T17:04:18.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 0x7f05b40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:18.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9195700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41037f0 0x7f05b4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:18.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9195700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41037f0 0x7f05b4075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:50180/0 (socket says 172.21.15.99:50180) 2024-05-14T17:04:19.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 0x7f05b40762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33518/0 (socket says 172.21.15.99:33518) 2024-05-14T17:04:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9195700 1 -- 172.21.15.99:0/2015811061 learned_addr learned my addr 172.21.15.99:0/2015811061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b8994700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41041a0 0x7f05b4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 -- 172.21.15.99:0/2015811061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41037f0 msgr2=0x7f05b4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:19.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41037f0 0x7f05b4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 -- 172.21.15.99:0/2015811061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41041a0 msgr2=0x7f05b4075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41041a0 0x7f05b4075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 -- 172.21.15.99:0/2015811061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b41a8020 con 0x7f05b41088f0 2024-05-14T17:04:19.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b8994700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41041a0 0x7f05b4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:19.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.696+0000 7f05b9996700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 0x7f05b40762c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00cc50 tx=0x7f05ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:19.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.697+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac016020 con 0x7f05b41088f0 2024-05-14T17:04:19.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.697+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b41a8310 con 0x7f05b41088f0 2024-05-14T17:04:19.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.697+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ac012420 con 0x7f05b41088f0 2024-05-14T17:04:19.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.697+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0115a0 con 0x7f05b41088f0 2024-05-14T17:04:19.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.697+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b41a8820 con 0x7f05b41088f0 2024-05-14T17:04:19.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.698+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b406a230 con 0x7f05b41088f0 2024-05-14T17:04:19.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.699+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f05ac019020 con 0x7f05b41088f0 2024-05-14T17:04:19.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.700+0000 7f05aa7fc700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f05a00b2b10 0x7f05a00b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:19.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.700+0000 7f05b9195700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f05a00b2b10 0x7f05a00b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:19.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.700+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f05ac0e2a10 con 0x7f05b41088f0 2024-05-14T17:04:19.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.701+0000 7f05b9195700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f05a00b2b10 0x7f05a00b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05b0005950 tx=0x7f05b000b500 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:19.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.705+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05ac020020 con 0x7f05b41088f0 2024-05-14T17:04:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.971+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05b402e560 con 0x7f05b41088f0 2024-05-14T17:04:19.248 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:18 smithi099 ceph-mon[31851]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-14T17:04:19.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.972+0000 7f05aa7fc700 1 -- 172.21.15.99:0/2015811061 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f05ac0aa2e0 con 0x7f05b41088f0 2024-05-14T17:04:19.262 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:04:19.269 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:04:19.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.974+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f05a00b2b10 msgr2=0x7f05a00b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:19.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.974+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f05a00b2b10 0x7f05a00b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05b0005950 tx=0x7f05b000b500 comp rx=0 tx=0).stop 2024-05-14T17:04:19.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.974+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 msgr2=0x7f05b40762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.974+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 0x7f05b40762c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00cc50 tx=0x7f05ac00cf60 comp rx=0 tx=0).stop 2024-05-14T17:04:19.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 shutdown_connections 2024-05-14T17:04:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f05a00b2b10 0x7f05a00b4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b41088f0 0x7f05b40762c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b41037f0 0x7f05b4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 --2- 172.21.15.99:0/2015811061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05b41041a0 0x7f05b4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:19.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 >> 172.21.15.99:0/2015811061 conn(0x7f05b407a6a0 msgr2=0x7f05b4102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:19.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 shutdown_connections 2024-05-14T17:04:19.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:18.975+0000 7f05bb3f9700 1 -- 172.21.15.99:0/2015811061 wait complete. 2024-05-14T17:04:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:04:19.465 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:18 smithi193 ceph-mon[35962]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-14T17:04:19.474 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:18 smithi150 ceph-mon[36204]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-14T17:04:19.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T17:00:11.851265+0000', 'modified': '2024-05-14T17:03:41.795311+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24463, 'mds_1': 24448, 'mds_2': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/1891176204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 1891176204}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 1891176204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3825469703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3825469703}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3825469703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/3113222034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 3113222034}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 3113222034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.150:6839/3105942045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 3105942045}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 3105942045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.193:6837/4135945134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 4135945134}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 4135945134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/4034311425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 4034311425}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 4034311425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24463, 'qdb_cluster': [24443, 24448, 24463]} 2024-05-14T17:04:19.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T17:04:19.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T17:04:19.977 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:20.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2015811061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:20.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2015811061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:20.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2015811061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 -- 172.21.15.99:0/1205492126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0102660 msgr2=0x7fcdc010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 --2- 172.21.15.99:0/1205492126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0102660 0x7fcdc010e770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009900 tx=0x7fcdb802a410 comp rx=0 tx=0).stop 2024-05-14T17:04:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 -- 172.21.15.99:0/1205492126 shutdown_connections 2024-05-14T17:04:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 --2- 172.21.15.99:0/1205492126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0102660 0x7fcdc010e770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 --2- 172.21.15.99:0/1205492126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 --2- 172.21.15.99:0/1205492126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdc01068f0 0x7fcdc0106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 -- 172.21.15.99:0/1205492126 >> 172.21.15.99:0/1205492126 conn(0x7fcdc0077750 msgr2=0x7fcdc0077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 -- 172.21.15.99:0/1205492126 shutdown_connections 2024-05-14T17:04:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.360+0000 7fcdc5047640 1 -- 172.21.15.99:0/1205492126 wait complete. 2024-05-14T17:04:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 Processor -- start 2024-05-14T17:04:20.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 -- start start 2024-05-14T17:04:20.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdc0102660 0x7fcdc019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdc01068f0 0x7fcdc01a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0111bd0 con 0x7fcdc0101ca0 2024-05-14T17:04:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0111a50 con 0x7fcdc0102660 2024-05-14T17:04:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdc5047640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc0111d50 con 0x7fcdc01068f0 2024-05-14T17:04:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33536/0 (socket says 172.21.15.99:33536) 2024-05-14T17:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.361+0000 7fcdbed9d640 1 -- 172.21.15.99:0/6420173 learned_addr learned my addr 172.21.15.99:0/6420173 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdc0102660 0x7fcdc019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbf59e640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdc01068f0 0x7fcdc01a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbed9d640 1 -- 172.21.15.99:0/6420173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdc01068f0 msgr2=0x7fcdc01a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdc01068f0 0x7fcdc01a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbed9d640 1 -- 172.21.15.99:0/6420173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdc0102660 msgr2=0x7fcdc019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdc0102660 0x7fcdc019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbed9d640 1 -- 172.21.15.99:0/6420173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc01a54d0 con 0x7fcdc0101ca0 2024-05-14T17:04:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0199f10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000ea80 tx=0x7fcdb000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdb7fff640 1 -- 172.21.15.99:0/6420173 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000ce40 con 0x7fcdc0101ca0 2024-05-14T17:04:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc01a56a0 con 0x7fcdc0101ca0 2024-05-14T17:04:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdb7fff640 1 -- 172.21.15.99:0/6420173 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb0005020 con 0x7fcdc0101ca0 2024-05-14T17:04:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdb7fff640 1 -- 172.21.15.99:0/6420173 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00116d0 con 0x7fcdc0101ca0 2024-05-14T17:04:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.362+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc01a5b70 con 0x7fcdc0101ca0 2024-05-14T17:04:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.364+0000 7fcdb7fff640 1 -- 172.21.15.99:0/6420173 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fcdb00054d0 con 0x7fcdc0101ca0 2024-05-14T17:04:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 Processor -- start 2024-05-14T17:04:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 -- start start 2024-05-14T17:04:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dcd4936c30 con 0x55dcd49c4210 2024-05-14T17:04:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcd49267d0 con 0x55dcd49cead0 2024-05-14T17:04:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dcd492a810 con 0x55dcd49c38f0 2024-05-14T17:04:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33548/0 (socket says 172.21.15.99:33548) 2024-05-14T17:04:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 -- 172.21.15.99:0/900225364 learned_addr learned my addr 172.21.15.99:0/900225364 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbf59e640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 -- 172.21.15.99:0/900225364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 msgr2=0x55dcd49c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 -- 172.21.15.99:0/900225364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 msgr2=0x55dcd49d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbed9d640 1 -- 172.21.15.99:0/900225364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd463b210 con 0x55dcd49c4210 2024-05-14T17:04:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdb7fff640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcdac0b2b30 0x7fcdac0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbf59e640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.365+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:21.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.366+0000 7fcdbf59e640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcdac0b2b30 0x7fcdac0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.366+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49ce590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00e0130 tx=0x7fcdb00a61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362bd5c017115b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.366+0000 7fcdb7fff640 1 -- 172.21.15.99:0/6420173 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fcdb00e2a20 con 0x7fcdc0101ca0 2024-05-14T17:04:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.366+0000 7fcdb57fa640 1 -- 172.21.15.99:0/900225364 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb001f020 con 0x55dcd49c4210 2024-05-14T17:04:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.366+0000 7fcdb57fa640 1 -- 172.21.15.99:0/900225364 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb00b2280 con 0x55dcd49c4210 2024-05-14T17:04:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.366+0000 7fcdb57fa640 1 -- 172.21.15.99:0/900225364 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00a94e0 con 0x55dcd49c4210 2024-05-14T17:04:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- 172.21.15.99:0/900225364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 msgr2=0x55dcd49ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49ce590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00e0130 tx=0x7fcdb00a61e0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdbf59e640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcdac0b2b30 0x7fcdac0b4ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8006010 tx=0x7fcdb80058d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- 172.21.15.99:0/900225364 shutdown_connections 2024-05-14T17:04:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49ce590 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/900225364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- 172.21.15.99:0/900225364 >> 172.21.15.99:0/900225364 conn(0x55dcd49bfe00 msgr2=0x55dcd49c01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- 172.21.15.99:0/900225364 shutdown_connections 2024-05-14T17:04:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- 172.21.15.99:0/900225364 wait complete. 2024-05-14T17:04:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- start start 2024-05-14T17:04:21.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 Processor -- start 2024-05-14T17:04:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dcd460c790 con 0x55dcd49c4210 2024-05-14T17:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcd48e2890 con 0x55dcd49cead0 2024-05-14T17:04:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dcd49d17a0 con 0x55dcd49c38f0 2024-05-14T17:04:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.367+0000 7fcdc7d64740 10 client.0 _ll_register_callbacks cb 0x55dcd48e5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33552/0 (socket says 172.21.15.99:33552) 2024-05-14T17:04:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 -- 172.21.15.99:0/1936391434 learned_addr learned my addr 172.21.15.99:0/1936391434 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbf59e640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 -- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 msgr2=0x55dcd49ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 -- 172.21.15.99:0/1936391434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 msgr2=0x55dcd49cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 -- 172.21.15.99:0/1936391434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd4707620 con 0x55dcd49c4210 2024-05-14T17:04:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49cb720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b8c0 tx=0x7fcda800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cf5b88b5caa84c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcda67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcda67fc640 1 -- 172.21.15.99:0/1936391434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8005020 con 0x55dcd49c4210 2024-05-14T17:04:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcda67fc640 1 -- 172.21.15.99:0/1936391434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fcda80051c0 con 0x55dcd49c4210 2024-05-14T17:04:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dcd4938b80 con 0x55dcd49c4210 2024-05-14T17:04:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcda67fc640 1 -- 172.21.15.99:0/1936391434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcda80053c0 con 0x55dcd49c4210 2024-05-14T17:04:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdc7d64740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dcd49dd190 con 0x55dcd49c4210 2024-05-14T17:04:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.368+0000 7fcda67fc640 1 -- 172.21.15.99:0/1936391434 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dcd4938b80 con 0x55dcd49c4210 2024-05-14T17:04:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcda67fc640 1 -- 172.21.15.99:0/1936391434 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcda8012780 con 0x55dcd49c4210 2024-05-14T17:04:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdc7d64740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdc7d64740 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdc7d64740 20 client.15315 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd49de6a0 0x55dcd49e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdc7d64740 4 client.15315 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcd4932360 con 0x55dcd49de6a0 2024-05-14T17:04:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd49de6a0 0x55dcd49e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcda67fc640 10 client.15315 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.369+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd49de6a0 0x55dcd49e0a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894d7d16576ac2bd server_cookie=c50a1c35215ea0ec in_seq=0 out_seq=0 2024-05-14T17:04:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcda67fc640 1 -- 172.21.15.99:0/1936391434 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcd4932360 con 0x55dcd49de6a0 2024-05-14T17:04:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcda67fc640 10 client.15315 handle_command_reply: tid=0 2024-05-14T17:04:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 client.15315 shutdown 2024-05-14T17:04:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 msgr2=0x55dcd49cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49cb720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b8c0 tx=0x7fcda800bd80 comp rx=0 tx=0).stop 2024-05-14T17:04:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 shutdown_connections 2024-05-14T17:04:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd49de6a0 0x55dcd49e0a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49cb720 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1936391434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c38f0 0x55dcd49ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcda67fc640 0 client.15315 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 >> 172.21.15.99:0/1936391434 conn(0x55dcd49bfe00 msgr2=0x55dcd49bf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 shutdown_connections 2024-05-14T17:04:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.370+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1936391434 wait complete. 2024-05-14T17:04:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.371+0000 7fcdc7d64740 20 client.15315 trim_cache size 0 max 16384 2024-05-14T17:04:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 Processor -- start 2024-05-14T17:04:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 -- start start 2024-05-14T17:04:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49c38f0 0x55dcd49bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49da600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49cead0 0x55dcd49cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01b0e10 con 0x55dcd49c4210 2024-05-14T17:04:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01ae6e0 con 0x55dcd49c38f0 2024-05-14T17:04:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01ac530 con 0x55dcd49cead0 2024-05-14T17:04:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49da600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33558/0 (socket says 172.21.15.99:33558) 2024-05-14T17:04:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 -- 172.21.15.99:0/3875774008 learned_addr learned my addr 172.21.15.99:0/3875774008 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49c38f0 0x55dcd49bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 -- 172.21.15.99:0/3875774008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49cead0 msgr2=0x55dcd49cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49cead0 0x55dcd49cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 -- 172.21.15.99:0/3875774008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49c38f0 msgr2=0x55dcd49bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49c38f0 0x55dcd49bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbe59c640 1 -- 172.21.15.99:0/3875774008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd49bfc30 con 0x55dcd49c4210 2024-05-14T17:04:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.386+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49c38f0 0x55dcd49bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.387+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49da600 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcda8004110 tx=0x7fcda8004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d826ddf3ded372bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.387+0000 7fcdb4ff9640 1 -- 172.21.15.99:0/3875774008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8027070 con 0x55dcd49c4210 2024-05-14T17:04:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.387+0000 7fcdb4ff9640 1 -- 172.21.15.99:0/3875774008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda8016070 con 0x55dcd49c4210 2024-05-14T17:04:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.387+0000 7fcdb4ff9640 1 -- 172.21.15.99:0/3875774008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda801a800 con 0x55dcd49c4210 2024-05-14T17:04:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.387+0000 7fcdc7d64740 1 -- 172.21.15.99:0/3875774008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 msgr2=0x55dcd49da600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.387+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49da600 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcda8004110 tx=0x7fcda8004140 comp rx=0 tx=0).stop 2024-05-14T17:04:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- 172.21.15.99:0/3875774008 shutdown_connections 2024-05-14T17:04:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49cead0 0x55dcd49cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c4210 0x55dcd49da600 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/3875774008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49c38f0 0x55dcd49bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- 172.21.15.99:0/3875774008 >> 172.21.15.99:0/3875774008 conn(0x55dcd49bfe00 msgr2=0x55dcd49d9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- 172.21.15.99:0/3875774008 shutdown_connections 2024-05-14T17:04:23.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- 172.21.15.99:0/3875774008 wait complete. 2024-05-14T17:04:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- start start 2024-05-14T17:04:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 Processor -- start 2024-05-14T17:04:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 0x55dcd49d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c4210 0x55dcd49d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01acba0 con 0x55dcd49c38f0 2024-05-14T17:04:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01ae010 con 0x55dcd49cead0 2024-05-14T17:04:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc01ad270 con 0x55dcd49c4210 2024-05-14T17:04:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdc7d64740 10 client.0 _ll_register_callbacks cb 0x7fcda80044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 0x55dcd49d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 0x55dcd49d4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33570/0 (socket says 172.21.15.99:33570) 2024-05-14T17:04:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.388+0000 7fcdbed9d640 1 -- 172.21.15.99:0/1580323464 learned_addr learned my addr 172.21.15.99:0/1580323464 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdbed9d640 1 -- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c4210 msgr2=0x55dcd49d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c4210 0x55dcd49d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdbed9d640 1 -- 172.21.15.99:0/1580323464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 msgr2=0x55dcd49d5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdbed9d640 1 -- 172.21.15.99:0/1580323464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd4a481f0 con 0x55dcd49c38f0 2024-05-14T17:04:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdbed9d640 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 0x55dcd49d4890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00e0130 tx=0x7fcdb0005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bac0956c8a92e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcda67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcda67fc640 1 -- 172.21.15.99:0/1580323464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb001f040 con 0x55dcd49c38f0 2024-05-14T17:04:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcda67fc640 1 -- 172.21.15.99:0/1580323464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fcdb00a9df0 con 0x55dcd49c38f0 2024-05-14T17:04:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda8016070 con 0x55dcd49c38f0 2024-05-14T17:04:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcda67fc640 1 -- 172.21.15.99:0/1580323464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcdb00a9300 con 0x55dcd49c38f0 2024-05-14T17:04:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dcd4a49470 con 0x55dcd49c38f0 2024-05-14T17:04:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcda67fc640 1 -- 172.21.15.99:0/1580323464 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda8016070 con 0x55dcd49c38f0 2024-05-14T17:04:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcda67fc640 1 -- 172.21.15.99:0/1580323464 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcdb000f040 con 0x55dcd49c38f0 2024-05-14T17:04:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 20 client.15321 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd4a49760 0x55dcd4a4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 4 client.15321 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.389+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcd48ac5f0 con 0x55dcd4a49760 2024-05-14T17:04:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.390+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd4a49760 0x55dcd4a4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.390+0000 7fcda67fc640 10 client.15321 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.390+0000 7fcdbe59c640 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd4a49760 0x55dcd4a4bb70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667c6007451c5de server_cookie=92e138d9b5345f39 in_seq=0 out_seq=0 2024-05-14T17:04:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcda67fc640 1 -- 172.21.15.99:0/1580323464 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dcd48ac5f0 con 0x55dcd4a49760 2024-05-14T17:04:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcda67fc640 10 client.15321 handle_command_reply: tid=0 2024-05-14T17:04:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 client.15321 shutdown 2024-05-14T17:04:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 msgr2=0x55dcd49d4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:23.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 0x55dcd49d4890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00e0130 tx=0x7fcdb0005ae0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 shutdown_connections 2024-05-14T17:04:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55dcd4a49760 0x55dcd4a4bb70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcd49cead0 0x55dcd49d5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcda67fc640 0 client.15321 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcd49c4210 0x55dcd49d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 --2- 172.21.15.99:0/1580323464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcd49c38f0 0x55dcd49d4890 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 >> 172.21.15.99:0/1580323464 conn(0x55dcd49bfe00 msgr2=0x55dcd49da600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 shutdown_connections 2024-05-14T17:04:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 1 -- 172.21.15.99:0/1580323464 wait complete. 2024-05-14T17:04:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.399+0000 7fcdc7d64740 20 client.15321 trim_cache size 0 max 16384 2024-05-14T17:04:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:04:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:04:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:04:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcdac0b2b30 msgr2=0x7fcdac0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcdac0b2b30 0x7fcdac0b4ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8006010 tx=0x7fcdb80058d0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 msgr2=0x7fcdc0199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0199f10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000ea80 tx=0x7fcdb000ef40 comp rx=0 tx=0).stop 2024-05-14T17:04:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 shutdown_connections 2024-05-14T17:04:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcdac0b2b30 0x7fcdac0b4ff0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdc01068f0 0x7fcdc01a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdc0102660 0x7fcdc019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 --2- 172.21.15.99:0/6420173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdc0101ca0 0x7fcdc0199f10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 >> 172.21.15.99:0/6420173 conn(0x7fcdc0077750 msgr2=0x7fcdc00ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:24.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 shutdown_connections 2024-05-14T17:04:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:20.400+0000 7fcdc5047640 1 -- 172.21.15.99:0/6420173 wait complete. 2024-05-14T17:04:24.041 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:20 smithi099 ceph-mon[31851]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-05-14T17:04:24.072 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:20 smithi193 ceph-mon[35962]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-05-14T17:04:24.082 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:22 smithi193 ceph-mon[35962]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-14T17:04:24.091 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:20 smithi150 ceph-mon[36204]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-05-14T17:04:24.101 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:22 smithi150 ceph-mon[36204]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-14T17:04:24.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 -- 172.21.15.99:0/635839785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22701095a0 msgr2=0x7f2270109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 --2- 172.21.15.99:0/635839785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22701095a0 0x7f2270109980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2264009990 tx=0x7f2264009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 -- 172.21.15.99:0/635839785 shutdown_connections 2024-05-14T17:04:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 --2- 172.21.15.99:0/635839785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f227006c470 0x7f22701103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 --2- 172.21.15.99:0/635839785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22701095a0 0x7f2270109980 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 --2- 172.21.15.99:0/635839785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006bab0 0x7f227006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 -- 172.21.15.99:0/635839785 >> 172.21.15.99:0/635839785 conn(0x7f22700793b0 msgr2=0x7f22700797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:24.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 -- 172.21.15.99:0/635839785 shutdown_connections 2024-05-14T17:04:24.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 -- 172.21.15.99:0/635839785 wait complete. 2024-05-14T17:04:24.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.559+0000 7f22771ac700 1 Processor -- start 2024-05-14T17:04:24.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 -- start start 2024-05-14T17:04:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 0x7f227019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006c470 0x7f227019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22701095a0 0x7f2270199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22701136d0 con 0x7f227006bab0 2024-05-14T17:04:24.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2270113550 con 0x7f227006c470 2024-05-14T17:04:24.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f22771ac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2270113850 con 0x7f22701095a0 2024-05-14T17:04:24.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 0x7f227019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 0x7f227019f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33578/0 (socket says 172.21.15.99:33578) 2024-05-14T17:04:24.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 -- 172.21.15.99:0/1737094708 learned_addr learned my addr 172.21.15.99:0/1737094708 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:24.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2275749700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22701095a0 0x7f2270199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f226ffff700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006c470 0x7f227019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 -- 172.21.15.99:0/1737094708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22701095a0 msgr2=0x7f2270199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22701095a0 0x7f2270199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 -- 172.21.15.99:0/1737094708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006c470 msgr2=0x7f227019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006c470 0x7f227019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 -- 172.21.15.99:0/1737094708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227019a1b0 con 0x7f227006bab0 2024-05-14T17:04:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2275749700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22701095a0 0x7f2270199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:24.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f226ffff700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006c470 0x7f227019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f2274f48700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 0x7f227019f890 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f22640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:24.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.560+0000 7f226dffb700 1 -- 172.21.15.99:0/1737094708 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264027070 con 0x7f227006bab0 2024-05-14T17:04:24.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.561+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227019a440 con 0x7f227006bab0 2024-05-14T17:04:24.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.561+0000 7f226dffb700 1 -- 172.21.15.99:0/1737094708 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2264003fa0 con 0x7f227006bab0 2024-05-14T17:04:24.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.561+0000 7f226dffb700 1 -- 172.21.15.99:0/1737094708 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264022db0 con 0x7f227006bab0 2024-05-14T17:04:24.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.561+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701a6760 con 0x7f227006bab0 2024-05-14T17:04:24.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.563+0000 7f226dffb700 1 -- 172.21.15.99:0/1737094708 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f226402a070 con 0x7f227006bab0 2024-05-14T17:04:24.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.564+0000 7f226dffb700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f22600b2b10 0x7f22600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.564+0000 7f226ffff700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f22600b2b10 0x7f22600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.564+0000 7f226dffb700 1 -- 172.21.15.99:0/1737094708 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f22640e3660 con 0x7f227006bab0 2024-05-14T17:04:24.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.564+0000 7f226ffff700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f22600b2b10 0x7f22600b4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f225c005950 tx=0x7f225c0058e0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:24.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.564+0000 7f2281a42b80 1 Processor -- start 2024-05-14T17:04:24.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.564+0000 7f2281a42b80 1 -- start start 2024-05-14T17:04:24.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x5584560596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x55845605dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2281a42b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558455fb7c40 con 0x558456059c10 2024-05-14T17:04:24.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2281a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558455fa77e0 con 0x55845605d910 2024-05-14T17:04:24.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2281a42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558455fab820 con 0x5584560592f0 2024-05-14T17:04:24.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33582/0 (socket says 172.21.15.99:33582) 2024-05-14T17:04:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 -- 172.21.15.99:0/1237144814 learned_addr learned my addr 172.21.15.99:0/1237144814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:24.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2275749700 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x55845605dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f2274f48700 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x5584560596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 -- 172.21.15.99:0/1237144814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 msgr2=0x5584560596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:24.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x5584560596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 -- 172.21.15.99:0/1237144814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 msgr2=0x55845605dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x55845605dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 -- 172.21.15.99:0/1237144814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558455ef7880 con 0x558456059c10 2024-05-14T17:04:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f226ffff700 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605d3d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f225c00fed0 tx=0x7f225c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6fb51a8df8aefe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:24.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f225affd700 1 -- 172.21.15.99:0/1237144814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c003880 con 0x558456059c10 2024-05-14T17:04:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f225affd700 1 -- 172.21.15.99:0/1237144814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f225c024070 con 0x558456059c10 2024-05-14T17:04:25.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.565+0000 7f225affd700 1 -- 172.21.15.99:0/1237144814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c01fd30 con 0x558456059c10 2024-05-14T17:04:25.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 -- 172.21.15.99:0/1237144814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 msgr2=0x55845605d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605d3d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f225c00fed0 tx=0x7f225c012000 comp rx=0 tx=0).stop 2024-05-14T17:04:25.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 -- 172.21.15.99:0/1237144814 shutdown_connections 2024-05-14T17:04:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x55845605dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x5584560596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1237144814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605d3d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 -- 172.21.15.99:0/1237144814 >> 172.21.15.99:0/1237144814 conn(0x558455f84200 msgr2=0x558456053930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:25.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 -- 172.21.15.99:0/1237144814 shutdown_connections 2024-05-14T17:04:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.566+0000 7f2281a42b80 1 -- 172.21.15.99:0/1237144814 wait complete. 2024-05-14T17:04:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 -- start start 2024-05-14T17:04:25.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 Processor -- start 2024-05-14T17:04:25.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x558456064da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x558456062140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x558456062680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558455eadf50 con 0x5584560592f0 2024-05-14T17:04:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558455f49770 con 0x558456059c10 2024-05-14T17:04:25.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558456065b70 con 0x55845605d910 2024-05-14T17:04:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2281a42b80 10 client.0 _ll_register_callbacks cb 0x558455f3e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:25.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2274f48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x558456064da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2274f48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x558456064da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33592/0 (socket says 172.21.15.99:33592) 2024-05-14T17:04:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2274f48700 1 -- 172.21.15.99:0/1367879460 learned_addr learned my addr 172.21.15.99:0/1367879460 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:25.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f2275749700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x558456062680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.567+0000 7f226ffff700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x558456062140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2274f48700 1 -- 172.21.15.99:0/1367879460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 msgr2=0x558456062680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2274f48700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x558456062680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2274f48700 1 -- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 msgr2=0x558456062140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2274f48700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x558456062140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2274f48700 1 -- 172.21.15.99:0/1367879460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558455f46640 con 0x5584560592f0 2024-05-14T17:04:25.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2275749700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x558456062680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2274f48700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x558456064da0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f226400c010 tx=0x7f2264022750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2f0720f641c75a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f224ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:25.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f224ffff700 1 -- 172.21.15.99:0/1367879460 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640aa7b0 con 0x5584560592f0 2024-05-14T17:04:25.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f224ffff700 1 -- 172.21.15.99:0/1367879460 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2264004150 con 0x5584560592f0 2024-05-14T17:04:25.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f225c024070 con 0x5584560592f0 2024-05-14T17:04:25.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f224ffff700 1 -- 172.21.15.99:0/1367879460 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22640aeb60 con 0x5584560592f0 2024-05-14T17:04:25.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2281a42b80 10 client.15333 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:25.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558456070250 con 0x5584560592f0 2024-05-14T17:04:25.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f224ffff700 1 -- 172.21.15.99:0/1367879460 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f225c024070 con 0x5584560592f0 2024-05-14T17:04:25.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f224ffff700 1 -- 172.21.15.99:0/1367879460 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f22640e0f40 con 0x5584560592f0 2024-05-14T17:04:25.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2281a42b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.568+0000 7f2281a42b80 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T17:04:25.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f2281a42b80 20 client.15333 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560728a0 0x558456073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f2281a42b80 4 client.15333 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:25.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 --> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558456025310 con 0x5584560728a0 2024-05-14T17:04:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f226ffff700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560728a0 0x558456073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f224ffff700 10 client.15333 ms_handle_connect on v2:172.21.15.150:6838/3105942045 2024-05-14T17:04:25.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.569+0000 7f226ffff700 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560728a0 0x558456073d10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a9e7a73aa79d306 server_cookie=d29b110aba8ef6b3 in_seq=0 out_seq=0 2024-05-14T17:04:25.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f224ffff700 1 -- 172.21.15.99:0/1367879460 <== mds.2 v2:172.21.15.150:6838/3105942045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558456025310 con 0x5584560728a0 2024-05-14T17:04:25.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f224ffff700 10 client.15333 handle_command_reply: tid=0 2024-05-14T17:04:25.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 client.15333 shutdown 2024-05-14T17:04:25.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 msgr2=0x558456064da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:25.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x558456064da0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f226400c010 tx=0x7f2264022750 comp rx=0 tx=0).stop 2024-05-14T17:04:25.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 shutdown_connections 2024-05-14T17:04:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560728a0 0x558456073d10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x558456062680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x558456064da0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f224ffff700 0 client.15333 ms_handle_reset on v2:172.21.15.150:6838/3105942045 2024-05-14T17:04:25.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1367879460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x558456062140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 >> 172.21.15.99:0/1367879460 conn(0x558455f84200 msgr2=0x5584560538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:25.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 shutdown_connections 2024-05-14T17:04:25.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 1 -- 172.21.15.99:0/1367879460 wait complete. 2024-05-14T17:04:26.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.570+0000 7f2281a42b80 20 client.15333 trim_cache size 0 max 16384 2024-05-14T17:04:26.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 Processor -- start 2024-05-14T17:04:26.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 -- start start 2024-05-14T17:04:26.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x5584560610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x55845604ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x55845607d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558456072050 con 0x5584560592f0 2024-05-14T17:04:26.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558456072720 con 0x558456059c10 2024-05-14T17:04:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2281a42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558456072df0 con 0x55845605d910 2024-05-14T17:04:26.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2274f48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x5584560610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2274f48700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x5584560610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33606/0 (socket says 172.21.15.99:33606) 2024-05-14T17:04:26.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2274f48700 1 -- 172.21.15.99:0/1735232952 learned_addr learned my addr 172.21.15.99:0/1735232952 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:26.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2274f48700 1 -- 172.21.15.99:0/1735232952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 msgr2=0x55845607d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f226ffff700 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x55845604ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2275749700 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x55845607d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.591+0000 7f2274f48700 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x55845607d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f2274f48700 1 -- 172.21.15.99:0/1735232952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 msgr2=0x55845604ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f2274f48700 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x55845604ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f2274f48700 1 -- 172.21.15.99:0/1735232952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845605d710 con 0x5584560592f0 2024-05-14T17:04:26.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f226ffff700 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x55845604ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:26.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f2274f48700 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x5584560610c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f22640c1040 tx=0x7f22640ab020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f329e244b96ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:26.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f225a7fc700 1 -- 172.21.15.99:0/1735232952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640aa7b0 con 0x5584560592f0 2024-05-14T17:04:26.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f225a7fc700 1 -- 172.21.15.99:0/1735232952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22640b1e50 con 0x5584560592f0 2024-05-14T17:04:26.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f225a7fc700 1 -- 172.21.15.99:0/1735232952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640e0f40 con 0x5584560592f0 2024-05-14T17:04:26.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f2281a42b80 1 -- 172.21.15.99:0/1735232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 msgr2=0x5584560610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:26.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.592+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x5584560610c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f22640c1040 tx=0x7f22640ab020 comp rx=0 tx=0).stop 2024-05-14T17:04:26.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- 172.21.15.99:0/1735232952 shutdown_connections 2024-05-14T17:04:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55845605d910 0x55845607d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584560592f0 0x5584560610c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 --2- 172.21.15.99:0/1735232952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558456059c10 0x55845604ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- 172.21.15.99:0/1735232952 >> 172.21.15.99:0/1735232952 conn(0x558455f84200 msgr2=0x558456058100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:26.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- 172.21.15.99:0/1735232952 shutdown_connections 2024-05-14T17:04:26.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- 172.21.15.99:0/1735232952 wait complete. 2024-05-14T17:04:26.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- start start 2024-05-14T17:04:26.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 Processor -- start 2024-05-14T17:04:26.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x55845605a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x5584560e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558456073b90 con 0x558456059c10 2024-05-14T17:04:26.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5584560734c0 con 0x55845605d910 2024-05-14T17:04:26.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558456074260 con 0x5584560592f0 2024-05-14T17:04:26.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.593+0000 7f2281a42b80 10 client.0 _ll_register_callbacks cb 0x558455f3e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:26.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33622/0 (socket says 172.21.15.99:33622) 2024-05-14T17:04:26.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 -- 172.21.15.99:0/2198579389 learned_addr learned my addr 172.21.15.99:0/2198579389 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f2275749700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x5584560e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f2274f48700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x55845605a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 -- 172.21.15.99:0/2198579389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 msgr2=0x55845605a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x55845605a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 -- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 msgr2=0x5584560e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x5584560e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 -- 172.21.15.99:0/2198579389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584560e2290 con 0x558456059c10 2024-05-14T17:04:26.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f226ffff700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605ae50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f225c00b410 tx=0x7f225c00fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124f21b5c920350b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f224ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:26.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f224ffff700 1 -- 172.21.15.99:0/2198579389 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c01fac0 con 0x558456059c10 2024-05-14T17:04:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f224ffff700 1 -- 172.21.15.99:0/2198579389 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f225c01fc60 con 0x558456059c10 2024-05-14T17:04:26.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22640b1e50 con 0x558456059c10 2024-05-14T17:04:26.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f224ffff700 1 -- 172.21.15.99:0/2198579389 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f225c01fe60 con 0x558456059c10 2024-05-14T17:04:26.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f2281a42b80 10 client.15339 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:26.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584560e3500 con 0x558456059c10 2024-05-14T17:04:26.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.594+0000 7f224ffff700 1 -- 172.21.15.99:0/2198579389 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22640b1e50 con 0x558456059c10 2024-05-14T17:04:26.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f224ffff700 1 -- 172.21.15.99:0/2198579389 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f225c0039e0 con 0x558456059c10 2024-05-14T17:04:26.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2281a42b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:26.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2281a42b80 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T17:04:26.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2281a42b80 20 client.15339 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:26.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2281a42b80 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560e7d90 0x5584560e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2281a42b80 4 client.15339 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:04:27.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 --> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] -- command(tid 0: {"prefix": "status"}) -- 0x558455fb9b90 con 0x5584560e7d90 2024-05-14T17:04:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2274f48700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560e7d90 0x5584560e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f224ffff700 10 client.15339 ms_handle_connect on v2:172.21.15.150:6838/3105942045 2024-05-14T17:04:27.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.595+0000 7f2274f48700 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560e7d90 0x5584560e6aa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=867927c35e609cd8 server_cookie=67ab974ac608179 in_seq=0 out_seq=0 2024-05-14T17:04:27.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f224ffff700 1 -- 172.21.15.99:0/2198579389 <== mds.2 v2:172.21.15.150:6838/3105942045 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558455fb9b90 con 0x5584560e7d90 2024-05-14T17:04:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f224ffff700 10 client.15339 handle_command_reply: tid=0 2024-05-14T17:04:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 client.15339 shutdown 2024-05-14T17:04:27.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 msgr2=0x55845605ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605ae50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f225c00b410 tx=0x7f225c00fa50 comp rx=0 tx=0).stop 2024-05-14T17:04:27.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 shutdown_connections 2024-05-14T17:04:27.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:6838/3105942045,v1:172.21.15.150:6839/3105942045] conn(0x5584560e7d90 0x5584560e6aa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55845605d910 0x5584560e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f224ffff700 0 client.15339 ms_handle_reset on v2:172.21.15.150:6838/3105942045 2024-05-14T17:04:27.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584560592f0 0x55845605a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 --2- 172.21.15.99:0/2198579389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558456059c10 0x55845605ae50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 >> 172.21.15.99:0/2198579389 conn(0x558455f84200 msgr2=0x55845605c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 shutdown_connections 2024-05-14T17:04:27.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 1 -- 172.21.15.99:0/2198579389 wait complete. 2024-05-14T17:04:27.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.596+0000 7f2281a42b80 20 client.15339 trim_cache size 0 max 16384 2024-05-14T17:04:27.261 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:04:27.272 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:04:27.291 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-05-14T17:04:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24443, 2024-05-14T17:04:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:04:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:04:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:04:27.423 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 235.48093649500001, 2024-05-14T17:04:27.431 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-05-14T17:04:27.441 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 83, 2024-05-14T17:04:27.454 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-05-14T17:04:27.471 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 267.06157189800001 2024-05-14T17:04:27.485 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:04:27.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:24 smithi193 ceph-mon[35962]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-05-14T17:04:27.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:26 smithi193 ceph-mon[35962]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-05-14T17:04:27.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:24 smithi150 ceph-mon[36204]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-05-14T17:04:27.540 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:26 smithi150 ceph-mon[36204]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-05-14T17:04:27.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f22600b2b10 msgr2=0x7f22600b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f22600b2b10 0x7f22600b4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f225c005950 tx=0x7f225c0058e0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 msgr2=0x7f227019f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 0x7f227019f890 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f22640047b0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 shutdown_connections 2024-05-14T17:04:27.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f22600b2b10 0x7f22600b4fd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22701095a0 0x7f2270199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227006bab0 0x7f227019f890 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 --2- 172.21.15.99:0/1737094708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227006c470 0x7f227019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 >> 172.21.15.99:0/1737094708 conn(0x7f22700793b0 msgr2=0x7f2270101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:27.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 shutdown_connections 2024-05-14T17:04:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:21.597+0000 7f22771ac700 1 -- 172.21.15.99:0/1737094708 wait complete. 2024-05-14T17:04:27.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-14T17:04:27.877 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:22 smithi099 ceph-mon[31851]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-14T17:04:27.886 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:24 smithi099 ceph-mon[31851]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-05-14T17:04:27.894 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:26 smithi099 ceph-mon[31851]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-05-14T17:04:28.159 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:29.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:28 smithi099 ceph-mon[31851]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-05-14T17:04:29.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:28 smithi150 ceph-mon[36204]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-05-14T17:04:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:28 smithi193 ceph-mon[35962]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-05-14T17:04:29.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 -- 172.21.15.99:0/3041776793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0103850 msgr2=0x7f10b0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:29.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 --2- 172.21.15.99:0/3041776793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0103850 0x7f10b0103c30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f10a00099c0 tx=0x7f10a0009cd0 comp rx=0 tx=0).stop 2024-05-14T17:04:29.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 -- 172.21.15.99:0/3041776793 shutdown_connections 2024-05-14T17:04:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 --2- 172.21.15.99:0/3041776793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10b0108920 0x7f10b01104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 --2- 172.21.15.99:0/3041776793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0103850 0x7f10b0103c30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 --2- 172.21.15.99:0/3041776793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0104200 0x7f10b01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 -- 172.21.15.99:0/3041776793 >> 172.21.15.99:0/3041776793 conn(0x7f10b007a620 msgr2=0x7f10b007aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 -- 172.21.15.99:0/3041776793 shutdown_connections 2024-05-14T17:04:30.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 -- 172.21.15.99:0/3041776793 wait complete. 2024-05-14T17:04:30.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.833+0000 7f10b69e1700 1 Processor -- start 2024-05-14T17:04:30.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 -- start start 2024-05-14T17:04:30.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10b0103850 0x7f10b0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 0x7f10b0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0108920 0x7f10b00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0113700 con 0x7f10b0104200 2024-05-14T17:04:30.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0113580 con 0x7f10b0108920 2024-05-14T17:04:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b69e1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0113880 con 0x7f10b0103850 2024-05-14T17:04:30.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 0x7f10b0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 0x7f10b0075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57522/0 (socket says 172.21.15.99:57522) 2024-05-14T17:04:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10af7fe700 1 -- 172.21.15.99:0/448672148 learned_addr learned my addr 172.21.15.99:0/448672148 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:30.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10affff700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10b0103850 0x7f10b0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.834+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0108920 0x7f10b00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10af7fe700 1 -- 172.21.15.99:0/448672148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10b0103850 msgr2=0x7f10b0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:30.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10af7fe700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10b0103850 0x7f10b0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10af7fe700 1 -- 172.21.15.99:0/448672148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0108920 msgr2=0x7f10b00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:30.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10af7fe700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0108920 0x7f10b00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10af7fe700 1 -- 172.21.15.99:0/448672148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b01a80e0 con 0x7f10b0104200 2024-05-14T17:04:30.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0108920 0x7f10b00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:30.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10af7fe700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 0x7f10b0075d80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f10a400d950 tx=0x7f10a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:30.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10ad7fa700 1 -- 172.21.15.99:0/448672148 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4016070 con 0x7f10b0104200 2024-05-14T17:04:30.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10ad7fa700 1 -- 172.21.15.99:0/448672148 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a4009df0 con 0x7f10b0104200 2024-05-14T17:04:30.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01a83d0 con 0x7f10b0104200 2024-05-14T17:04:30.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10ad7fa700 1 -- 172.21.15.99:0/448672148 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a400f3c0 con 0x7f10b0104200 2024-05-14T17:04:30.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.835+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a8910 con 0x7f10b0104200 2024-05-14T17:04:30.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.838+0000 7f10ad7fa700 1 -- 172.21.15.99:0/448672148 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f10a4019070 con 0x7f10b0104200 2024-05-14T17:04:30.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.839+0000 7f10ad7fa700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f10980b2b10 0x7f10980b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.839+0000 7f10affff700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f10980b2b10 0x7f10980b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.839+0000 7f10ad7fa700 1 -- 172.21.15.99:0/448672148 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f10a40e24c0 con 0x7f10b0104200 2024-05-14T17:04:30.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.839+0000 7f10c1277b80 1 Processor -- start 2024-05-14T17:04:30.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.839+0000 7f10c1277b80 1 -- start start 2024-05-14T17:04:30.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.839+0000 7f10affff700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f10980b2b10 0x7f10980b4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009990 tx=0x7f10a000b790 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:30.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee8662f0 0x5583ee8666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee866c10 0x5583ee86a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee86ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10c1277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583ee7c4c40 con 0x5583ee86a910 2024-05-14T17:04:30.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10c1277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5583ee7b47e0 con 0x5583ee8662f0 2024-05-14T17:04:30.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10c1277b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583ee7b8820 con 0x5583ee866c10 2024-05-14T17:04:30.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee86ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee86ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57530/0 (socket says 172.21.15.99:57530) 2024-05-14T17:04:30.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 -- 172.21.15.99:0/4036054727 learned_addr learned my addr 172.21.15.99:0/4036054727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:30.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10af7fe700 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee866c10 0x5583ee86a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10affff700 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee8662f0 0x5583ee8666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 -- 172.21.15.99:0/4036054727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee866c10 msgr2=0x5583ee86a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee866c10 0x5583ee86a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 -- 172.21.15.99:0/4036054727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee8662f0 msgr2=0x5583ee8666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee8662f0 0x5583ee8666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 -- 172.21.15.99:0/4036054727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ee704880 con 0x5583ee86a910 2024-05-14T17:04:30.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10affff700 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee8662f0 0x5583ee8666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:30.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee86ad50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f10a80085a0 tx=0x7f10a80088b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ce72bad4659fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:30.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10967fc700 1 -- 172.21.15.99:0/4036054727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8003c00 con 0x5583ee86a910 2024-05-14T17:04:30.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10967fc700 1 -- 172.21.15.99:0/4036054727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8003da0 con 0x5583ee86a910 2024-05-14T17:04:30.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.840+0000 7f10967fc700 1 -- 172.21.15.99:0/4036054727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8012910 con 0x5583ee86a910 2024-05-14T17:04:30.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 -- 172.21.15.99:0/4036054727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 msgr2=0x5583ee86ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee86ad50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f10a80085a0 tx=0x7f10a80088b0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 -- 172.21.15.99:0/4036054727 shutdown_connections 2024-05-14T17:04:30.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee86ad50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee8662f0 0x5583ee8666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 --2- 172.21.15.99:0/4036054727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee866c10 0x5583ee86a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 -- 172.21.15.99:0/4036054727 >> 172.21.15.99:0/4036054727 conn(0x5583ee791200 msgr2=0x5583ee860930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:30.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 -- 172.21.15.99:0/4036054727 shutdown_connections 2024-05-14T17:04:30.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.841+0000 7f10c1277b80 1 -- 172.21.15.99:0/4036054727 wait complete. 2024-05-14T17:04:30.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 -- start start 2024-05-14T17:04:30.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 Processor -- start 2024-05-14T17:04:30.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee871c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 0x5583ee86f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee86a910 0x5583ee86f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583ee6baf50 con 0x5583ee866c10 2024-05-14T17:04:31.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5583ee756770 con 0x5583ee86a910 2024-05-14T17:04:31.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583ee872b70 con 0x5583ee8662f0 2024-05-14T17:04:31.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10c1277b80 10 client.0 _ll_register_callbacks cb 0x5583ee74b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:31.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 0x5583ee86f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10affff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee871c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 0x5583ee86f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57534/0 (socket says 172.21.15.99:57534) 2024-05-14T17:04:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10af7fe700 1 -- 172.21.15.99:0/219869293 learned_addr learned my addr 172.21.15.99:0/219869293 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.842+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee86a910 0x5583ee86f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10af7fe700 1 -- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 msgr2=0x5583ee871c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10af7fe700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee871c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10af7fe700 1 -- 172.21.15.99:0/219869293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee86a910 msgr2=0x5583ee86f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10af7fe700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee86a910 0x5583ee86f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10af7fe700 1 -- 172.21.15.99:0/219869293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ee753640 con 0x5583ee866c10 2024-05-14T17:04:31.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee86a910 0x5583ee86f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:31.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10af7fe700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 0x5583ee86f0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10a40a9510 tx=0x7f10a40a9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf5a1259c7a887c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:31.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f108f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:31.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f108f7fe700 1 -- 172.21.15.99:0/219869293 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4020040 con 0x5583ee866c10 2024-05-14T17:04:31.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f108f7fe700 1 -- 172.21.15.99:0/219869293 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f10a40b1f50 con 0x5583ee866c10 2024-05-14T17:04:31.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a8003da0 con 0x5583ee866c10 2024-05-14T17:04:31.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f108f7fe700 1 -- 172.21.15.99:0/219869293 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f10a406d050 con 0x5583ee866c10 2024-05-14T17:04:31.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10c1277b80 10 client.15351 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:31.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583ee87d1b0 con 0x5583ee866c10 2024-05-14T17:04:31.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f108f7fe700 1 -- 172.21.15.99:0/219869293 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8003da0 con 0x5583ee866c10 2024-05-14T17:04:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f108f7fe700 1 -- 172.21.15.99:0/219869293 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f10a40dfda0 con 0x5583ee866c10 2024-05-14T17:04:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10c1277b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:31.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10c1277b80 10 client.15351 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T17:04:31.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.843+0000 7f10c1277b80 20 client.15351 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:31.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.844+0000 7f10c1277b80 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee87f860 0x5583ee880cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.844+0000 7f10c1277b80 4 client.15351 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:31.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.844+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 --> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583ee832310 con 0x5583ee87f860 2024-05-14T17:04:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.844+0000 7f10affff700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee87f860 0x5583ee880cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.844+0000 7f108f7fe700 10 client.15351 ms_handle_connect on v2:172.21.15.193:6836/4135945134 2024-05-14T17:04:31.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.844+0000 7f10affff700 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee87f860 0x5583ee880cd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4a207ce00d86834 server_cookie=2153ce9984f2ea5b in_seq=0 out_seq=0 2024-05-14T17:04:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f108f7fe700 1 -- 172.21.15.99:0/219869293 <== mds.1 v2:172.21.15.193:6836/4135945134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583ee832310 con 0x5583ee87f860 2024-05-14T17:04:31.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f108f7fe700 10 client.15351 handle_command_reply: tid=0 2024-05-14T17:04:31.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 client.15351 shutdown 2024-05-14T17:04:31.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 msgr2=0x5583ee86f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:31.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 0x5583ee86f0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10a40a9510 tx=0x7f10a40a9d70 comp rx=0 tx=0).stop 2024-05-14T17:04:31.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 shutdown_connections 2024-05-14T17:04:31.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee87f860 0x5583ee880cd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee86a910 0x5583ee86f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee871c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 --2- 172.21.15.99:0/219869293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee866c10 0x5583ee86f0a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 >> 172.21.15.99:0/219869293 conn(0x5583ee791200 msgr2=0x5583ee86bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:31.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f108f7fe700 0 client.15351 ms_handle_reset on v2:172.21.15.193:6836/4135945134 2024-05-14T17:04:31.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 shutdown_connections 2024-05-14T17:04:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 1 -- 172.21.15.99:0/219869293 wait complete. 2024-05-14T17:04:31.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.845+0000 7f10c1277b80 20 client.15351 trim_cache size 0 max 16384 2024-05-14T17:04:31.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 Processor -- start 2024-05-14T17:04:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 -- start start 2024-05-14T17:04:31.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 0x5583ee871540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee86fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee86a910 0x5583ee85ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583ee87f010 con 0x5583ee8662f0 2024-05-14T17:04:31.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5583ee87f6e0 con 0x5583ee866c10 2024-05-14T17:04:31.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10c1277b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583ee87fdb0 con 0x5583ee86a910 2024-05-14T17:04:31.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 0x5583ee871540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 0x5583ee871540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57536/0 (socket says 172.21.15.99:57536) 2024-05-14T17:04:31.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 -- 172.21.15.99:0/1002549575 learned_addr learned my addr 172.21.15.99:0/1002549575 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:31.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee86a910 0x5583ee85ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 -- 172.21.15.99:0/1002549575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee86a910 msgr2=0x5583ee85ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10af7fe700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee86fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee86a910 0x5583ee85ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 -- 172.21.15.99:0/1002549575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 msgr2=0x5583ee86fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.866+0000 7f10affff700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee86fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10affff700 1 -- 172.21.15.99:0/1002549575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ee87be10 con 0x5583ee8662f0 2024-05-14T17:04:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee86a910 0x5583ee85ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:31.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10af7fe700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee86fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10affff700 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 0x5583ee871540 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10a0003800 tx=0x7f10a0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c44e8516ece414d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:31.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f1095ffb700 1 -- 172.21.15.99:0/1002549575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0024a60 con 0x5583ee8662f0 2024-05-14T17:04:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f1095ffb700 1 -- 172.21.15.99:0/1002549575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a0024c00 con 0x5583ee8662f0 2024-05-14T17:04:31.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f1095ffb700 1 -- 172.21.15.99:0/1002549575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a000f460 con 0x5583ee8662f0 2024-05-14T17:04:31.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 -- 172.21.15.99:0/1002549575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 msgr2=0x5583ee871540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 0x5583ee871540 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10a0003800 tx=0x7f10a0023000 comp rx=0 tx=0).stop 2024-05-14T17:04:32.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 -- 172.21.15.99:0/1002549575 shutdown_connections 2024-05-14T17:04:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee86a910 0x5583ee85ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee8662f0 0x5583ee871540 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 --2- 172.21.15.99:0/1002549575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee86fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 -- 172.21.15.99:0/1002549575 >> 172.21.15.99:0/1002549575 conn(0x5583ee791200 msgr2=0x5583ee85a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:32.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.867+0000 7f10c1277b80 1 -- 172.21.15.99:0/1002549575 shutdown_connections 2024-05-14T17:04:32.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 -- 172.21.15.99:0/1002549575 wait complete. 2024-05-14T17:04:32.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 -- start start 2024-05-14T17:04:32.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 Processor -- start 2024-05-14T17:04:32.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee88a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee88a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee884830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583ee880b50 con 0x5583ee86a910 2024-05-14T17:04:32.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5583ee880480 con 0x5583ee866c10 2024-05-14T17:04:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583ee881220 con 0x5583ee8662f0 2024-05-14T17:04:32.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10c1277b80 10 client.0 _ll_register_callbacks cb 0x5583ee74b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:32.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10b4f7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee884830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee88a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.868+0000 7f10b4f7e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee884830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57550/0 (socket says 172.21.15.99:57550) 2024-05-14T17:04:32.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 -- 172.21.15.99:0/120031710 learned_addr learned my addr 172.21.15.99:0/120031710 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:32.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10affff700 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee88a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 -- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 msgr2=0x5583ee88a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee88a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 -- 172.21.15.99:0/120031710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 msgr2=0x5583ee88a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee88a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 -- 172.21.15.99:0/120031710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ee885040 con 0x5583ee86a910 2024-05-14T17:04:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10b4f7e700 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee884830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f10a8016040 tx=0x7f10a8004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa10796d97249de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:32.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f108f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:32.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f108f7fe700 1 -- 172.21.15.99:0/120031710 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8023020 con 0x5583ee86a910 2024-05-14T17:04:32.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f108f7fe700 1 -- 172.21.15.99:0/120031710 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f10a8012af0 con 0x5583ee86a910 2024-05-14T17:04:32.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a0024c00 con 0x5583ee86a910 2024-05-14T17:04:32.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f108f7fe700 1 -- 172.21.15.99:0/120031710 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f10a802e050 con 0x5583ee86a910 2024-05-14T17:04:32.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10c1277b80 10 client.15357 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:32.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583ee8eb940 con 0x5583ee86a910 2024-05-14T17:04:32.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.869+0000 7f108f7fe700 1 -- 172.21.15.99:0/120031710 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a0024c00 con 0x5583ee86a910 2024-05-14T17:04:32.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f108f7fe700 1 -- 172.21.15.99:0/120031710 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f10a8013070 con 0x5583ee86a910 2024-05-14T17:04:32.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:32.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 10 client.15357 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-14T17:04:32.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 20 client.15357 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:32.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee8f0260 0x5583ee8eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 4 client.15357 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:04:32.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 --> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] -- command(tid 0: {"prefix": "status"}) -- 0x5583ee7c6b90 con 0x5583ee8f0260 2024-05-14T17:04:32.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10affff700 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee8f0260 0x5583ee8eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f108f7fe700 10 client.15357 ms_handle_connect on v2:172.21.15.193:6836/4135945134 2024-05-14T17:04:32.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10affff700 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee8f0260 0x5583ee8eef70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5fb13fa55d061e4 server_cookie=3c935392665ae148 in_seq=0 out_seq=0 2024-05-14T17:04:32.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f108f7fe700 1 -- 172.21.15.99:0/120031710 <== mds.1 v2:172.21.15.193:6836/4135945134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5583ee7c6b90 con 0x5583ee8f0260 2024-05-14T17:04:32.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f108f7fe700 10 client.15357 handle_command_reply: tid=0 2024-05-14T17:04:32.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.870+0000 7f10c1277b80 1 client.15357 shutdown 2024-05-14T17:04:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 msgr2=0x5583ee884830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee884830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f10a8016040 tx=0x7f10a8004f90 comp rx=0 tx=0).stop 2024-05-14T17:04:32.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 shutdown_connections 2024-05-14T17:04:32.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:6836/4135945134,v1:172.21.15.193:6837/4135945134] conn(0x5583ee8f0260 0x5583ee8eef70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583ee86a910 0x5583ee884830 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ee8662f0 0x5583ee88a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f108f7fe700 0 client.15357 ms_handle_reset on v2:172.21.15.193:6836/4135945134 2024-05-14T17:04:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 --2- 172.21.15.99:0/120031710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5583ee866c10 0x5583ee88a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 >> 172.21.15.99:0/120031710 conn(0x5583ee791200 msgr2=0x5583ee87b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:32.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 shutdown_connections 2024-05-14T17:04:32.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 1 -- 172.21.15.99:0/120031710 wait complete. 2024-05-14T17:04:32.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10c1277b80 20 client.15357 trim_cache size 0 max 16384 2024-05-14T17:04:32.771 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:04:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:04:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-05-14T17:04:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24448, 2024-05-14T17:04:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:04:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:04:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:04:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 244.76181652400001, 2024-05-14T17:04:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-05-14T17:04:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 83, 2024-05-14T17:04:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-05-14T17:04:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 290.30235783199998 2024-05-14T17:04:32.950 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:04:32.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f10980b2b10 msgr2=0x7f10980b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:32.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10b69e1700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f10980b2b10 0x7f10980b4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009990 tx=0x7f10a000b790 comp rx=0 tx=0).stop 2024-05-14T17:04:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 msgr2=0x7f10b0075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.871+0000 7f10b69e1700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 0x7f10b0075d80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f10a400d950 tx=0x7f10a400dc60 comp rx=0 tx=0).stop 2024-05-14T17:04:33.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 shutdown_connections 2024-05-14T17:04:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f10980b2b10 0x7f10980b4fd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b0108920 0x7f10b00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10b0103850 0x7f10b0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 --2- 172.21.15.99:0/448672148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10b0104200 0x7f10b0075d80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 >> 172.21.15.99:0/448672148 conn(0x7f10b007a620 msgr2=0x7f10b0101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:33.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 shutdown_connections 2024-05-14T17:04:33.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:29.872+0000 7f10b69e1700 1 -- 172.21.15.99:0/448672148 wait complete. 2024-05-14T17:04:33.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:30 smithi193 ceph-mon[35962]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-14T17:04:33.150 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:30 smithi150 ceph-mon[36204]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-14T17:04:33.178 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T17:04:33.185 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:30 smithi099 ceph-mon[31851]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-05-14T17:04:33.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:32 smithi150 ceph-mon[36204]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-14T17:04:33.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:32 smithi099 ceph-mon[31851]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-14T17:04:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:32 smithi193 ceph-mon[35962]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-14T17:04:33.448 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- 172.21.15.99:0/3612184015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 msgr2=0x7f16f4107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- 172.21.15.99:0/3612184015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f4107d20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009930 tx=0x7f16dc02a420 comp rx=0 tx=0).stop 2024-05-14T17:04:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- 172.21.15.99:0/3612184015 shutdown_connections 2024-05-14T17:04:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- 172.21.15.99:0/3612184015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4102340 0x7f16f410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- 172.21.15.99:0/3612184015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f4101980 0x7f16f4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- 172.21.15.99:0/3612184015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f4107d20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- 172.21.15.99:0/3612184015 >> 172.21.15.99:0/3612184015 conn(0x7f16f4078ad0 msgr2=0x7f16f4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- 172.21.15.99:0/3612184015 shutdown_connections 2024-05-14T17:04:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- 172.21.15.99:0/3612184015 wait complete. 2024-05-14T17:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 Processor -- start 2024-05-14T17:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- start start 2024-05-14T17:04:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4101980 0x7f16f41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f4102340 0x7f16f4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4111c00 con 0x7f16f4107940 2024-05-14T17:04:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4111a80 con 0x7f16f4101980 2024-05-14T17:04:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.497+0000 7f16faf24640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4111d80 con 0x7f16f4102340 2024-05-14T17:04:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f419da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57578/0 (socket says 172.21.15.99:57578) 2024-05-14T17:04:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 -- 172.21.15.99:0/2036226392 learned_addr learned my addr 172.21.15.99:0/2036226392 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f3fff640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f4102340 0x7f16f4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4101980 0x7f16f41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 -- 172.21.15.99:0/2036226392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f4102340 msgr2=0x7f16f4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f4102340 0x7f16f4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 -- 172.21.15.99:0/2036226392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4101980 msgr2=0x7f16f41991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4101980 0x7f16f41991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 -- 172.21.15.99:0/2036226392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f419e1c0 con 0x7f16f4107940 2024-05-14T17:04:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4101980 0x7f16f41991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f94c1640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f419da90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00eaa0 tx=0x7f16ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f1ffb640 1 -- 172.21.15.99:0/2036226392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00cdb0 con 0x7f16f4107940 2024-05-14T17:04:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f1ffb640 1 -- 172.21.15.99:0/2036226392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ec005020 con 0x7f16f4107940 2024-05-14T17:04:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f419e4b0 con 0x7f16f4107940 2024-05-14T17:04:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16f1ffb640 1 -- 172.21.15.99:0/2036226392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0116d0 con 0x7f16f4107940 2024-05-14T17:04:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.498+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f41a5d40 con 0x7f16f4107940 2024-05-14T17:04:34.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.500+0000 7f16f1ffb640 1 -- 172.21.15.99:0/2036226392 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f16ec0051d0 con 0x7f16f4107940 2024-05-14T17:04:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 Processor -- start 2024-05-14T17:04:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 -- start start 2024-05-14T17:04:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 0x55900c222cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c223210 0x55900c22d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c22fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55900c195c30 con 0x55900c2228f0 2024-05-14T17:04:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55900c1857d0 con 0x55900c22dad0 2024-05-14T17:04:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16fdc43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55900c189810 con 0x55900c223210 2024-05-14T17:04:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 0x55900c222cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 0x55900c222cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57580/0 (socket says 172.21.15.99:57580) 2024-05-14T17:04:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16f3fff640 1 -- 172.21.15.99:0/3490871492 learned_addr learned my addr 172.21.15.99:0/3490871492 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16f1ffb640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b2bb0 0x7f16e00b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c223210 0x55900c22d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.501+0000 7f16f94c1640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b2bb0 0x7f16e00b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f3fff640 1 -- 172.21.15.99:0/3490871492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c223210 msgr2=0x55900c22d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f3fff640 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c223210 0x55900c22d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f3fff640 1 -- 172.21.15.99:0/3490871492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 msgr2=0x55900c22fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f3fff640 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c22fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f3fff640 1 -- 172.21.15.99:0/3490871492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900be9a210 con 0x55900c2228f0 2024-05-14T17:04:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f1ffb640 1 -- 172.21.15.99:0/2036226392 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f16ec0e2a00 con 0x7f16f4107940 2024-05-14T17:04:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f3fff640 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 0x55900c222cd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16e400b790 tx=0x7f16e400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0ad5b5b27e5563 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16f94c1640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b2bb0 0x7f16e00b5070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0e0000 tx=0x7f16ec076390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16eaffd640 1 -- 172.21.15.99:0/3490871492 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e400bea0 con 0x55900c2228f0 2024-05-14T17:04:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16eaffd640 1 -- 172.21.15.99:0/3490871492 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e4016030 con 0x55900c2228f0 2024-05-14T17:04:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.502+0000 7f16eaffd640 1 -- 172.21.15.99:0/3490871492 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4004c60 con 0x55900c2228f0 2024-05-14T17:04:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 -- 172.21.15.99:0/3490871492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 msgr2=0x55900c222cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 0x55900c222cd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16e400b790 tx=0x7f16e400bc50 comp rx=0 tx=0).stop 2024-05-14T17:04:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 -- 172.21.15.99:0/3490871492 shutdown_connections 2024-05-14T17:04:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c22fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c223210 0x55900c22d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 --2- 172.21.15.99:0/3490871492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c2228f0 0x55900c222cd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 -- 172.21.15.99:0/3490871492 >> 172.21.15.99:0/3490871492 conn(0x55900c21ee00 msgr2=0x55900c21f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 -- 172.21.15.99:0/3490871492 shutdown_connections 2024-05-14T17:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 -- 172.21.15.99:0/3490871492 wait complete. 2024-05-14T17:04:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 -- start start 2024-05-14T17:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 Processor -- start 2024-05-14T17:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.503+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c22d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c22a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c22ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55900be6b790 con 0x55900c223210 2024-05-14T17:04:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55900c141890 con 0x55900c2228f0 2024-05-14T17:04:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55900c230740 con 0x55900c22dad0 2024-05-14T17:04:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 10 client.0 _ll_register_callbacks cb 0x55900c144d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c22a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c22a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57584/0 (socket says 172.21.15.99:57584) 2024-05-14T17:04:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 -- 172.21.15.99:0/305925828 learned_addr learned my addr 172.21.15.99:0/305925828 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f3fff640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c22d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f3fff640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c22ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 -- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 msgr2=0x55900c22ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c22ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 -- 172.21.15.99:0/305925828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 msgr2=0x55900c22d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c22d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 -- 172.21.15.99:0/305925828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900bf66620 con 0x55900c223210 2024-05-14T17:04:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c22a700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005ab0 tx=0x7f16dc031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287f8a6ef8b57ded server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16d3fff640 1 -- 172.21.15.99:0/305925828 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03f070 con 0x55900c223210 2024-05-14T17:04:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16d3fff640 1 -- 172.21.15.99:0/305925828 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f16dc004250 con 0x55900c223210 2024-05-14T17:04:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e4016030 con 0x55900c223210 2024-05-14T17:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16d3fff640 1 -- 172.21.15.99:0/305925828 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16dc035070 con 0x55900c223210 2024-05-14T17:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 10 client.15369 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.504+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55900c23c180 con 0x55900c223210 2024-05-14T17:04:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16d3fff640 1 -- 172.21.15.99:0/305925828 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e4016030 con 0x55900c223210 2024-05-14T17:04:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16d3fff640 1 -- 172.21.15.99:0/305925828 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f16dc004a60 con 0x55900c223210 2024-05-14T17:04:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16fdc43740 10 client.15369 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16fdc43740 10 client.15369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16fdc43740 20 client.15369 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16fdc43740 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c23d690 0x55900c23fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16fdc43740 4 client.15369 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900c197b80 con 0x55900c23d690 2024-05-14T17:04:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16f3fff640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c23d690 0x55900c23fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16d3fff640 10 client.15369 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.505+0000 7f16f3fff640 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c23d690 0x55900c23fa80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5dd17ebd529c805 server_cookie=387034bdbb6d361d in_seq=0 out_seq=0 2024-05-14T17:04:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16d3fff640 1 -- 172.21.15.99:0/305925828 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55900c197b80 con 0x55900c23d690 2024-05-14T17:04:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16d3fff640 10 client.15369 handle_command_reply: tid=0 2024-05-14T17:04:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16fdc43740 1 client.15369 shutdown 2024-05-14T17:04:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 msgr2=0x55900c22a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16fdc43740 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c22a700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005ab0 tx=0x7f16dc031750 comp rx=0 tx=0).stop 2024-05-14T17:04:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 shutdown_connections 2024-05-14T17:04:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.506+0000 7f16fdc43740 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c23d690 0x55900c23fa80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c22ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c22a700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16d3fff640 0 client.15369 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 1 --2- 172.21.15.99:0/305925828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c22d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 >> 172.21.15.99:0/305925828 conn(0x55900c21ee00 msgr2=0x55900c21e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 shutdown_connections 2024-05-14T17:04:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 1 -- 172.21.15.99:0/305925828 wait complete. 2024-05-14T17:04:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.507+0000 7f16fdc43740 20 client.15369 trim_cache size 0 max 16384 2024-05-14T17:04:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 Processor -- start 2024-05-14T17:04:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 -- start start 2024-05-14T17:04:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c239610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c245350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c245890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41b0ed0 con 0x55900c223210 2024-05-14T17:04:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41ae7a0 con 0x55900c2228f0 2024-05-14T17:04:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16fdc43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41ac5f0 con 0x55900c22dad0 2024-05-14T17:04:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c245350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c245350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57586/0 (socket says 172.21.15.99:57586) 2024-05-14T17:04:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 -- 172.21.15.99:0/2267089280 learned_addr learned my addr 172.21.15.99:0/2267089280 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f3fff640 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c239610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 -- 172.21.15.99:0/2267089280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 msgr2=0x55900c245890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c245890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 -- 172.21.15.99:0/2267089280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 msgr2=0x55900c239610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c239610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f8cc0640 1 -- 172.21.15.99:0/2267089280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900c21ec30 con 0x55900c223210 2024-05-14T17:04:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.522+0000 7f16f3fff640 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c239610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c245350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16dc004e80 tx=0x7f16dc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470598f8e4e6f26d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16ea7fc640 1 -- 172.21.15.99:0/2267089280 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc050040 con 0x55900c223210 2024-05-14T17:04:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16ea7fc640 1 -- 172.21.15.99:0/2267089280 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16dc004250 con 0x55900c223210 2024-05-14T17:04:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16ea7fc640 1 -- 172.21.15.99:0/2267089280 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03f040 con 0x55900c223210 2024-05-14T17:04:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 -- 172.21.15.99:0/2267089280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 msgr2=0x55900c245350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c245350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16dc004e80 tx=0x7f16dc02a420 comp rx=0 tx=0).stop 2024-05-14T17:04:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 -- 172.21.15.99:0/2267089280 shutdown_connections 2024-05-14T17:04:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c22dad0 0x55900c245890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c245350 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 --2- 172.21.15.99:0/2267089280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c2228f0 0x55900c239610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 -- 172.21.15.99:0/2267089280 >> 172.21.15.99:0/2267089280 conn(0x55900c21ee00 msgr2=0x55900c21e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 -- 172.21.15.99:0/2267089280 shutdown_connections 2024-05-14T17:04:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.523+0000 7f16fdc43740 1 -- 172.21.15.99:0/2267089280 wait complete. 2024-05-14T17:04:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 -- start start 2024-05-14T17:04:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 Processor -- start 2024-05-14T17:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c2228f0 0x55900c236040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c2333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c233b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41acc60 con 0x55900c223210 2024-05-14T17:04:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41ae0d0 con 0x55900c22dad0 2024-05-14T17:04:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41ad330 con 0x55900c2228f0 2024-05-14T17:04:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16fdc43740 10 client.0 _ll_register_callbacks cb 0x7f16dc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c2333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c2333d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57588/0 (socket says 172.21.15.99:57588) 2024-05-14T17:04:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 -- 172.21.15.99:0/603080823 learned_addr learned my addr 172.21.15.99:0/603080823 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f3fff640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c233b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 -- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c2228f0 msgr2=0x55900c236040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c2228f0 0x55900c236040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 -- 172.21.15.99:0/603080823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 msgr2=0x55900c233b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c233b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.524+0000 7f16f8cc0640 1 -- 172.21.15.99:0/603080823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900c2a6f10 con 0x55900c223210 2024-05-14T17:04:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16f3fff640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c233b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16f8cc0640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c2333d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f16dc02a940 tx=0x7f16dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ac745b9967cf2d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16d3fff640 1 -- 172.21.15.99:0/603080823 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03f040 con 0x55900c223210 2024-05-14T17:04:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16d3fff640 1 -- 172.21.15.99:0/603080823 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f16dc004ba0 con 0x55900c223210 2024-05-14T17:04:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16dc004250 con 0x55900c223210 2024-05-14T17:04:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16d3fff640 1 -- 172.21.15.99:0/603080823 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16dc050040 con 0x55900c223210 2024-05-14T17:04:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 10 client.15375 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55900c2a7ff0 con 0x55900c223210 2024-05-14T17:04:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16d3fff640 1 -- 172.21.15.99:0/603080823 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16dc004250 con 0x55900c223210 2024-05-14T17:04:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16d3fff640 1 -- 172.21.15.99:0/603080823 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f16dc003b00 con 0x55900c223210 2024-05-14T17:04:36.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 10 client.15375 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 20 client.15375 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c2a8600 0x55900c2aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 4 client.15375 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:36.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.525+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55900c191360 con 0x55900c2a8600 2024-05-14T17:04:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.526+0000 7f16f3fff640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c2a8600 0x55900c2aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.526+0000 7f16f3fff640 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c2a8600 0x55900c2aaa20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ec201994aeff18 server_cookie=b9bb2e12f8b1d632 in_seq=0 out_seq=0 2024-05-14T17:04:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.526+0000 7f16d3fff640 10 client.15375 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16d3fff640 1 -- 172.21.15.99:0/603080823 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55900c191360 con 0x55900c2a8600 2024-05-14T17:04:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16d3fff640 10 client.15375 handle_command_reply: tid=0 2024-05-14T17:04:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 client.15375 shutdown 2024-05-14T17:04:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 msgr2=0x55900c2333d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c2333d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f16dc02a940 tx=0x7f16dc0046c0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 shutdown_connections 2024-05-14T17:04:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55900c2a8600 0x55900c2aaa20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55900c22dad0 0x55900c233b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900c223210 0x55900c2333d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16d3fff640 0 client.15375 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 --2- 172.21.15.99:0/603080823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55900c2228f0 0x55900c236040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 >> 172.21.15.99:0/603080823 conn(0x55900c21ee00 msgr2=0x55900c238600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 shutdown_connections 2024-05-14T17:04:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 1 -- 172.21.15.99:0/603080823 wait complete. 2024-05-14T17:04:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.532+0000 7f16fdc43740 20 client.15375 trim_cache size 0 max 16384 2024-05-14T17:04:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:04:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:04:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:04:36.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:34 smithi193 ceph-mon[35962]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-14T17:04:36.366 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:34 smithi150 ceph-mon[36204]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-14T17:04:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b2bb0 msgr2=0x7f16e00b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b2bb0 0x7f16e00b5070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0e0000 tx=0x7f16ec076390 comp rx=0 tx=0).stop 2024-05-14T17:04:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 msgr2=0x7f16f419da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f419da90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00eaa0 tx=0x7f16ec00ef60 comp rx=0 tx=0).stop 2024-05-14T17:04:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 shutdown_connections 2024-05-14T17:04:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b2bb0 0x7f16e00b5070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f4107940 0x7f16f419da90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f4102340 0x7f16f4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 --2- 172.21.15.99:0/2036226392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f4101980 0x7f16f41991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 >> 172.21.15.99:0/2036226392 conn(0x7f16f4078ad0 msgr2=0x7f16f406a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 shutdown_connections 2024-05-14T17:04:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:33.533+0000 7f16faf24640 1 -- 172.21.15.99:0/2036226392 wait complete. 2024-05-14T17:04:36.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:34 smithi099 ceph-mon[31851]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-14T17:04:36.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.141+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/618647077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca0108470 msgr2=0x7f8ca0108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.141+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/618647077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca0108470 0x7f8ca0108850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009990 tx=0x7f8c9c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/618647077 shutdown_connections 2024-05-14T17:04:36.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/618647077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca0102da0 0x7f8ca010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/618647077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca0108470 0x7f8ca0108850 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/618647077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca01023e0 0x7f8ca0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/618647077 >> 172.21.15.99:0/618647077 conn(0x7f8ca00fdef0 msgr2=0x7f8ca0100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:36.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/618647077 shutdown_connections 2024-05-14T17:04:36.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/618647077 wait complete. 2024-05-14T17:04:36.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.142+0000 7f8ca6f9e700 1 Processor -- start 2024-05-14T17:04:36.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 -- start start 2024-05-14T17:04:36.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 0x7f8ca0104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:36.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca0102da0 0x7f8ca0104580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:36.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca0108470 0x7f8ca0104ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0112620 con 0x7f8ca01023e0 2024-05-14T17:04:36.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01124a0 con 0x7f8ca0102da0 2024-05-14T17:04:36.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca6f9e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca01127a0 con 0x7f8ca0108470 2024-05-14T17:04:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 0x7f8ca0104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:36.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 0x7f8ca0104040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40618/0 (socket says 172.21.15.99:40618) 2024-05-14T17:04:36.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2975440198 learned_addr learned my addr 172.21.15.99:0/2975440198 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:36.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca579b700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca0102da0 0x7f8ca0104580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:36.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca679d700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca0108470 0x7f8ca0104ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2975440198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca0108470 msgr2=0x7f8ca0104ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:36.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca0108470 0x7f8ca0104ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:36.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2975440198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca0102da0 msgr2=0x7f8ca0104580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:37.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca0102da0 0x7f8ca0104580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2975440198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca01a8140 con 0x7f8ca01023e0 2024-05-14T17:04:37.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.143+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 0x7f8ca0104040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:37.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.144+0000 7f8c96ffd700 1 -- 172.21.15.99:0/2975440198 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c027070 con 0x7f8ca01023e0 2024-05-14T17:04:37.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.144+0000 7f8c96ffd700 1 -- 172.21.15.99:0/2975440198 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c003f20 con 0x7f8ca01023e0 2024-05-14T17:04:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.144+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca01a83d0 con 0x7f8ca01023e0 2024-05-14T17:04:37.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.144+0000 7f8c96ffd700 1 -- 172.21.15.99:0/2975440198 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c022ca0 con 0x7f8ca01023e0 2024-05-14T17:04:37.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.144+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca01a8820 con 0x7f8ca01023e0 2024-05-14T17:04:37.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.146+0000 7f8c96ffd700 1 -- 172.21.15.99:0/2975440198 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f8c9c02a070 con 0x7f8ca01023e0 2024-05-14T17:04:37.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.147+0000 7f8c96ffd700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8c8c0b2b10 0x7f8c8c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.147+0000 7f8ca579b700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8c8c0b2b10 0x7f8c8c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.147+0000 7f8c96ffd700 1 -- 172.21.15.99:0/2975440198 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f8c9c0e36b0 con 0x7f8ca01023e0 2024-05-14T17:04:37.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.147+0000 7f8cb2a96b80 1 Processor -- start 2024-05-14T17:04:37.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.147+0000 7f8ca579b700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8c8c0b2b10 0x7f8c8c0b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ca006cb00 tx=0x7f8c9000b3c0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:37.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 -- start start 2024-05-14T17:04:37.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 0x55d013499ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d013494360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013494980 0x55d013494dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d0133f2c40 con 0x55d013499900 2024-05-14T17:04:37.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d0133e27e0 con 0x55d013494980 2024-05-14T17:04:37.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d0133e6820 con 0x55d01349a220 2024-05-14T17:04:37.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 0x55d013499ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 0x55d013499ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40622/0 (socket says 172.21.15.99:40622) 2024-05-14T17:04:37.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/55097212 learned_addr learned my addr 172.21.15.99:0/55097212 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca579b700 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d013494360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca679d700 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013494980 0x55d013494dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/55097212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 msgr2=0x55d013494360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:37.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d013494360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/55097212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013494980 msgr2=0x55d013494dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:37.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013494980 0x55d013494dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/55097212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d013332880 con 0x55d013499900 2024-05-14T17:04:37.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca679d700 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013494980 0x55d013494dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:37.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.148+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 0x55d013499ce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0a6880 tx=0x7f8c9c0aa310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785744b77165873a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:37.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8c8bfff700 1 -- 172.21.15.99:0/55097212 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0aa800 con 0x55d013499900 2024-05-14T17:04:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8c8bfff700 1 -- 172.21.15.99:0/55097212 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0aebb0 con 0x55d013499900 2024-05-14T17:04:37.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8c8bfff700 1 -- 172.21.15.99:0/55097212 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0b3230 con 0x55d013499900 2024-05-14T17:04:37.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/55097212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 msgr2=0x55d013499ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:37.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 0x55d013499ce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0a6880 tx=0x7f8c9c0aa310 comp rx=0 tx=0).stop 2024-05-14T17:04:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/55097212 shutdown_connections 2024-05-14T17:04:37.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013494980 0x55d013494dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013499900 0x55d013499ce0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/55097212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d013494360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.149+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/55097212 >> 172.21.15.99:0/55097212 conn(0x55d0133bf200 msgr2=0x55d01348eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:37.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/55097212 shutdown_connections 2024-05-14T17:04:37.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/55097212 wait complete. 2024-05-14T17:04:37.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 -- start start 2024-05-14T17:04:37.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 Processor -- start 2024-05-14T17:04:37.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 0x55d01349ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d01349d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d01349d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d0132e8f50 con 0x55d013494980 2024-05-14T17:04:37.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d013384770 con 0x55d013499900 2024-05-14T17:04:37.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.150+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d0134a0b00 con 0x55d01349a220 2024-05-14T17:04:37.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8cb2a96b80 10 client.0 _ll_register_callbacks cb 0x55d013379530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 0x55d01349ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 0x55d01349ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40626/0 (socket says 172.21.15.99:40626) 2024-05-14T17:04:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2248057915 learned_addr learned my addr 172.21.15.99:0/2248057915 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca679d700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d01349d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca579b700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d01349d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 msgr2=0x55d01349d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:37.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d01349d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2248057915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 msgr2=0x55d01349d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:37.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d01349d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 -- 172.21.15.99:0/2248057915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d013381640 con 0x55d013494980 2024-05-14T17:04:37.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 0x55d01349ffa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0e1430 tx=0x7f8c9c022680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63699e1dc8727b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8c897fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:38.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8c897fa700 1 -- 172.21.15.99:0/2248057915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0b10e0 con 0x55d013494980 2024-05-14T17:04:38.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8c897fa700 1 -- 172.21.15.99:0/2248057915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f8c9c0e11a0 con 0x55d013494980 2024-05-14T17:04:38.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c0aebb0 con 0x55d013494980 2024-05-14T17:04:38.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8c897fa700 1 -- 172.21.15.99:0/2248057915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c0aa800 con 0x55d013494980 2024-05-14T17:04:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8cb2a96b80 10 client.15387 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:38.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.151+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0134ab0b0 con 0x55d013494980 2024-05-14T17:04:38.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8c897fa700 1 -- 172.21.15.99:0/2248057915 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0aebb0 con 0x55d013494980 2024-05-14T17:04:38.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8c897fa700 1 -- 172.21.15.99:0/2248057915 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9c092110 con 0x55d013494980 2024-05-14T17:04:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8cb2a96b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8cb2a96b80 10 client.15387 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T17:04:38.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8cb2a96b80 20 client.15387 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:38.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d0134ad410 0x55d0134ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8cb2a96b80 4 client.15387 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d013460310 con 0x55d0134ad410 2024-05-14T17:04:38.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.152+0000 7f8ca579b700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d0134ad410 0x55d0134ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8ca579b700 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d0134ad410 0x55d0134ae880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991e06d2202e18f4 server_cookie=51696c78fafca015 in_seq=0 out_seq=0 2024-05-14T17:04:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8c897fa700 10 client.15387 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:38.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8c897fa700 1 -- 172.21.15.99:0/2248057915 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d013460310 con 0x55d0134ad410 2024-05-14T17:04:38.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8c897fa700 10 client.15387 handle_command_reply: tid=0 2024-05-14T17:04:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8cb2a96b80 1 client.15387 shutdown 2024-05-14T17:04:38.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 msgr2=0x55d01349ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:38.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.153+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 0x55d01349ffa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0e1430 tx=0x7f8c9c022680 comp rx=0 tx=0).stop 2024-05-14T17:04:38.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 shutdown_connections 2024-05-14T17:04:38.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d01349a220 0x55d01349d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d0134ad410 0x55d0134ae880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d013494980 0x55d01349ffa0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/2248057915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d01349d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8c897fa700 0 client.15387 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 >> 172.21.15.99:0/2248057915 conn(0x55d0133bf200 msgr2=0x55d01348e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 shutdown_connections 2024-05-14T17:04:38.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/2248057915 wait complete. 2024-05-14T17:04:38.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.154+0000 7f8cb2a96b80 20 client.15387 trim_cache size 0 max 16384 2024-05-14T17:04:38.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.174+0000 7f8cb2a96b80 1 Processor -- start 2024-05-14T17:04:38.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 -- start start 2024-05-14T17:04:38.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d013494360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d0134acbc0 con 0x55d01349a220 2024-05-14T17:04:38.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d0134ad290 con 0x55d013499900 2024-05-14T17:04:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d0134ad960 con 0x55d013494980 2024-05-14T17:04:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40642/0 (socket says 172.21.15.99:40642) 2024-05-14T17:04:38.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 -- 172.21.15.99:0/3694762181 learned_addr learned my addr 172.21.15.99:0/3694762181 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:38.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d013494360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca579b700 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 -- 172.21.15.99:0/3694762181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 msgr2=0x55d013494360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d013494360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 -- 172.21.15.99:0/3694762181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 msgr2=0x55d0134a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:38.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 -- 172.21.15.99:0/3694762181 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d013497d80 con 0x55d01349a220 2024-05-14T17:04:38.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8ca679d700 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134a99a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800a750 tx=0x7f8c9800a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae06992c2e4c8a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8c8b7fe700 1 -- 172.21.15.99:0/3694762181 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9801b020 con 0x55d01349a220 2024-05-14T17:04:38.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8c8b7fe700 1 -- 172.21.15.99:0/3694762181 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c98017af0 con 0x55d01349a220 2024-05-14T17:04:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.175+0000 7f8c8b7fe700 1 -- 172.21.15.99:0/3694762181 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98016b50 con 0x55d01349a220 2024-05-14T17:04:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/3694762181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 msgr2=0x55d0134a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:38.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134a99a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800a750 tx=0x7f8c9800a780 comp rx=0 tx=0).stop 2024-05-14T17:04:38.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/3694762181 shutdown_connections 2024-05-14T17:04:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134a99a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d013494360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/3694762181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/3694762181 >> 172.21.15.99:0/3694762181 conn(0x55d0133bf200 msgr2=0x55d0134a8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:38.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/3694762181 shutdown_connections 2024-05-14T17:04:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/3694762181 wait complete. 2024-05-14T17:04:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.176+0000 7f8cb2a96b80 1 -- start start 2024-05-14T17:04:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 Processor -- start 2024-05-14T17:04:38.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d0134b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d0134ae700 con 0x55d01349a220 2024-05-14T17:04:39.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d0134ae030 con 0x55d013499900 2024-05-14T17:04:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d0134aedd0 con 0x55d013494980 2024-05-14T17:04:39.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8cb2a96b80 10 client.0 _ll_register_callbacks cb 0x55d013379530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40652/0 (socket says 172.21.15.99:40652) 2024-05-14T17:04:39.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 -- 172.21.15.99:0/1926013940 learned_addr learned my addr 172.21.15.99:0/1926013940 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:39.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca579b700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d0134b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 -- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 msgr2=0x55d0134b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:39.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d0134b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 -- 172.21.15.99:0/1926013940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 msgr2=0x55d0134b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca679d700 1 -- 172.21.15.99:0/1926013940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0134b4fc0 con 0x55d01349a220 2024-05-14T17:04:39.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d0134b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.177+0000 7f8ca579b700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:39.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8ca679d700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134b4980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c98004c20 tx=0x7f8c98017440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840281ec7137e6d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:39.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8c897fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8c897fa700 1 -- 172.21.15.99:0/1926013940 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98016990 con 0x55d01349a220 2024-05-14T17:04:39.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8c897fa700 1 -- 172.21.15.99:0/1926013940 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f8c98016b30 con 0x55d01349a220 2024-05-14T17:04:39.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c98017af0 con 0x55d01349a220 2024-05-14T17:04:39.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8c897fa700 1 -- 172.21.15.99:0/1926013940 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8c98016d50 con 0x55d01349a220 2024-05-14T17:04:39.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 10 client.15393 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:39.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d013521ed0 con 0x55d01349a220 2024-05-14T17:04:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8c897fa700 1 -- 172.21.15.99:0/1926013940 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c98017af0 con 0x55d01349a220 2024-05-14T17:04:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8c897fa700 1 -- 172.21.15.99:0/1926013940 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9802b420 con 0x55d01349a220 2024-05-14T17:04:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:39.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 10 client.15393 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T17:04:39.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 20 client.15393 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:39.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d013526720 0x55d013525430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 4 client.15393 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T17:04:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.178+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "status"}) -- 0x55d0133f4b90 con 0x55d013526720 2024-05-14T17:04:39.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d013526720 0x55d013525430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8c897fa700 10 client.15393 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8ca5f9c700 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d013526720 0x55d013525430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec266f6bee2378b server_cookie=c0b07c42029d0738 in_seq=0 out_seq=0 2024-05-14T17:04:39.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8c897fa700 1 -- 172.21.15.99:0/1926013940 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d0133f4b90 con 0x55d013526720 2024-05-14T17:04:39.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8c897fa700 10 client.15393 handle_command_reply: tid=0 2024-05-14T17:04:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8cb2a96b80 1 client.15393 shutdown 2024-05-14T17:04:39.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 msgr2=0x55d0134b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.179+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134b4980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c98004c20 tx=0x7f8c98017440 comp rx=0 tx=0).stop 2024-05-14T17:04:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 shutdown_connections 2024-05-14T17:04:39.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d013526720 0x55d013525430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d01349a220 0x55d0134b4980 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8c897fa700 0 client.15393 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d013494980 0x55d0134b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 --2- 172.21.15.99:0/1926013940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d013499900 0x55d0134b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 >> 172.21.15.99:0/1926013940 conn(0x55d0133bf200 msgr2=0x55d013497600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:39.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 shutdown_connections 2024-05-14T17:04:39.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 1 -- 172.21.15.99:0/1926013940 wait complete. 2024-05-14T17:04:39.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8cb2a96b80 20 client.15393 trim_cache size 0 max 16384 2024-05-14T17:04:39.733 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:04:39.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "2a006d80-1212-11ef-bc98-c7b262605968", 2024-05-14T17:04:39.814 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-05-14T17:04:39.843 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24463, 2024-05-14T17:04:39.850 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-05-14T17:04:39.858 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-05-14T17:04:39.869 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-05-14T17:04:39.889 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 263.12598422899998, 2024-05-14T17:04:39.898 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-05-14T17:04:39.906 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 83, 2024-05-14T17:04:39.915 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-05-14T17:04:39.928 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 290.208512939 2024-05-14T17:04:39.939 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:04:39.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8c8c0b2b10 msgr2=0x7f8c8c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8c8c0b2b10 0x7f8c8c0b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ca006cb00 tx=0x7f8c9000b3c0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 msgr2=0x7f8ca0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 0x7f8ca0104040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c0047b0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 shutdown_connections 2024-05-14T17:04:40.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8c8c0b2b10 0x7f8c8c0b4fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ca0108470 0x7f8ca0104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca01023e0 0x7f8ca0104040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 --2- 172.21.15.99:0/2975440198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ca0102da0 0x7f8ca0104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 >> 172.21.15.99:0/2975440198 conn(0x7f8ca00fdef0 msgr2=0x7f8ca0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:40.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 shutdown_connections 2024-05-14T17:04:40.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:35.180+0000 7f8ca6f9e700 1 -- 172.21.15.99:0/2975440198 wait complete. 2024-05-14T17:04:40.208 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-14T17:04:40.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:36 smithi099 ceph-mon[31851]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1022 B/s wr, 6 op/s 2024-05-14T17:04:40.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:38 smithi099 ceph-mon[31851]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T17:04:40.243 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:36 smithi193 ceph-mon[35962]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1022 B/s wr, 6 op/s 2024-05-14T17:04:40.249 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:38 smithi193 ceph-mon[35962]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T17:04:40.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:36 smithi150 ceph-mon[36204]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1022 B/s wr, 6 op/s 2024-05-14T17:04:40.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:38 smithi150 ceph-mon[36204]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T17:04:41.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:40 smithi150 ceph-mon[36204]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T17:04:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:40 smithi099 ceph-mon[31851]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T17:04:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:40 smithi193 ceph-mon[35962]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T17:04:43.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:42 smithi193 ceph-mon[35962]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T17:04:43.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:42 smithi150 ceph-mon[36204]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T17:04:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:42 smithi099 ceph-mon[31851]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-14T17:04:45.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:44 smithi150 ceph-mon[36204]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T17:04:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:44 smithi099 ceph-mon[31851]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T17:04:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:44 smithi193 ceph-mon[35962]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-14T17:04:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.632+0000 7f54dd6fc640 1 -- 172.21.15.99:0/996409337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 msgr2=0x7f54d81040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.632+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/996409337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d81040c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009900 tx=0x7f54c802a480 comp rx=0 tx=0).stop 2024-05-14T17:04:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- 172.21.15.99:0/996409337 shutdown_connections 2024-05-14T17:04:46.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/996409337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54d810d0b0 0x7f54d810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/996409337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8104600 0x7f54d810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/996409337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d81040c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- 172.21.15.99:0/996409337 >> 172.21.15.99:0/996409337 conn(0x7f54d8078a60 msgr2=0x7f54d8078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- 172.21.15.99:0/996409337 shutdown_connections 2024-05-14T17:04:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- 172.21.15.99:0/996409337 wait complete. 2024-05-14T17:04:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 Processor -- start 2024-05-14T17:04:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- start start 2024-05-14T17:04:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54d8104600 0x7f54d819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d810d0b0 0x7f54d819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8112cf0 con 0x7f54d8103ce0 2024-05-14T17:04:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8112b70 con 0x7f54d810d0b0 2024-05-14T17:04:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.633+0000 7f54dd6fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8112e70 con 0x7f54d8104600 2024-05-14T17:04:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38640/0 (socket says 172.21.15.99:38640) 2024-05-14T17:04:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3314748243 learned_addr learned my addr 172.21.15.99:0/3314748243 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54cffff640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54d8104600 0x7f54d819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d77fe640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d810d0b0 0x7f54d819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3314748243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54d8104600 msgr2=0x7f54d819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54d8104600 0x7f54d819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3314748243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d810d0b0 msgr2=0x7f54d819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d810d0b0 0x7f54d819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3314748243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d81a03c0 con 0x7f54d8103ce0 2024-05-14T17:04:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d77fe640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d810d0b0 0x7f54d819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d819b3d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005940 tx=0x7f54c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d4ff9640 1 -- 172.21.15.99:0/3314748243 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c803f030 con 0x7f54d8103ce0 2024-05-14T17:04:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d4ff9640 1 -- 172.21.15.99:0/3314748243 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8003aa0 con 0x7f54d8103ce0 2024-05-14T17:04:47.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d81a0650 con 0x7f54d8103ce0 2024-05-14T17:04:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54d4ff9640 1 -- 172.21.15.99:0/3314748243 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8035070 con 0x7f54d8103ce0 2024-05-14T17:04:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.634+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d81a7f40 con 0x7f54d8103ce0 2024-05-14T17:04:47.259 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:46 smithi150 ceph-mon[36204]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:04:47.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:46 smithi193 ceph-mon[35962]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:04:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54d4ff9640 1 -- 172.21.15.99:0/3314748243 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f54c8042030 con 0x7f54d8103ce0 2024-05-14T17:04:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 Processor -- start 2024-05-14T17:04:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 -- start start 2024-05-14T17:04:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1846cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 0x5565c1851590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c1853ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565c17b9c30 con 0x5565c1847210 2024-05-14T17:04:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565c17a97d0 con 0x5565c1851ad0 2024-05-14T17:04:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54e0413740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5565c17ad810 con 0x5565c18468f0 2024-05-14T17:04:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.637+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 0x5565c1851590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 0x5565c1851590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38650/0 (socket says 172.21.15.99:38650) 2024-05-14T17:04:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 -- 172.21.15.99:0/1658549243 learned_addr learned my addr 172.21.15.99:0/1658549243 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54cffff640 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c1853ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d77fe640 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1846cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d4ff9640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f54c40b2b30 0x7f54c40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 -- 172.21.15.99:0/1658549243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 msgr2=0x5565c1846cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1846cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 -- 172.21.15.99:0/1658549243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 msgr2=0x5565c1853ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c1853ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 -- 172.21.15.99:0/1658549243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c14be210 con 0x5565c1847210 2024-05-14T17:04:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d77fe640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f54c40b2b30 0x7f54c40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 0x5565c1851590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54c81005e0 tx=0x7f54c80c6770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42741e35a10766ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d4ff9640 1 -- 172.21.15.99:0/3314748243 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f54c8102ed0 con 0x7f54d8103ce0 2024-05-14T17:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54cd7fa640 1 -- 172.21.15.99:0/1658549243 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8049070 con 0x5565c1847210 2024-05-14T17:04:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54cd7fa640 1 -- 172.21.15.99:0/1658549243 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c80c9b70 con 0x5565c1847210 2024-05-14T17:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54cd7fa640 1 -- 172.21.15.99:0/1658549243 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80c9e30 con 0x5565c1847210 2024-05-14T17:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.638+0000 7f54d77fe640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f54c40b2b30 0x7f54c40b4ff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f54d806bcf0 tx=0x7f54d0004a40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 -- 172.21.15.99:0/1658549243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 msgr2=0x5565c1851590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 0x5565c1851590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54c81005e0 tx=0x7f54c80c6770 comp rx=0 tx=0).stop 2024-05-14T17:04:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 -- 172.21.15.99:0/1658549243 shutdown_connections 2024-05-14T17:04:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c1853ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1847210 0x5565c1851590 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 --2- 172.21.15.99:0/1658549243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1846cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 -- 172.21.15.99:0/1658549243 >> 172.21.15.99:0/1658549243 conn(0x5565c1842e00 msgr2=0x5565c18431e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 -- 172.21.15.99:0/1658549243 shutdown_connections 2024-05-14T17:04:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.639+0000 7f54e0413740 1 -- 172.21.15.99:0/1658549243 wait complete. 2024-05-14T17:04:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 -- start start 2024-05-14T17:04:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 Processor -- start 2024-05-14T17:04:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 0x5565c1851410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c1847210 0x5565c184e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c184ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565c148f790 con 0x5565c18468f0 2024-05-14T17:04:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565c1765890 con 0x5565c1851ad0 2024-05-14T17:04:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5565c18544c0 con 0x5565c1847210 2024-05-14T17:04:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54e0413740 10 client.0 _ll_register_callbacks cb 0x5565c1768d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 0x5565c1851410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 0x5565c1851410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38658/0 (socket says 172.21.15.99:38658) 2024-05-14T17:04:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3483503704 learned_addr learned my addr 172.21.15.99:0/3483503704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54cffff640 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c1847210 0x5565c184e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c1847210 msgr2=0x5565c184e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c1847210 0x5565c184e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3483503704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 msgr2=0x5565c184ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c184ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 -- 172.21.15.99:0/3483503704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c158a620 con 0x5565c18468f0 2024-05-14T17:04:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 0x5565c1851410 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f54c8031440 tx=0x7f54c8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24cd3f83ab0e7f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.640+0000 7f54be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54be7fc640 1 -- 172.21.15.99:0/3483503704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8049070 con 0x5565c18468f0 2024-05-14T17:04:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54be7fc640 1 -- 172.21.15.99:0/3483503704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f54c80c62f0 con 0x5565c18468f0 2024-05-14T17:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c80c9b70 con 0x5565c18468f0 2024-05-14T17:04:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54be7fc640 1 -- 172.21.15.99:0/3483503704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54c80c8430 con 0x5565c18468f0 2024-05-14T17:04:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565c185ff30 con 0x5565c18468f0 2024-05-14T17:04:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54be7fc640 1 -- 172.21.15.99:0/3483503704 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c80c9b70 con 0x5565c18468f0 2024-05-14T17:04:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54be7fc640 1 -- 172.21.15.99:0/3483503704 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f54c80c59d0 con 0x5565c18468f0 2024-05-14T17:04:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 20 client.15405 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c1861390 0x5565c1863800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 4 client.15405 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.641+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565c17bbb80 con 0x5565c1861390 2024-05-14T17:04:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.642+0000 7f54cffff640 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c1861390 0x5565c1863800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.642+0000 7f54cffff640 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c1861390 0x5565c1863800 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a754de9acc9ad9f server_cookie=7172e9e798ead008 in_seq=0 out_seq=0 2024-05-14T17:04:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.642+0000 7f54be7fc640 10 client.15405 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54be7fc640 1 -- 172.21.15.99:0/3483503704 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565c17bbb80 con 0x5565c1861390 2024-05-14T17:04:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54be7fc640 10 client.15405 handle_command_reply: tid=0 2024-05-14T17:04:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 client.15405 shutdown 2024-05-14T17:04:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 msgr2=0x5565c1851410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 0x5565c1851410 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f54c8031440 tx=0x7f54c8003f10 comp rx=0 tx=0).stop 2024-05-14T17:04:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 shutdown_connections 2024-05-14T17:04:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c1861390 0x5565c1863800 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1851ad0 0x5565c184ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c1847210 0x5565c184e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 --2- 172.21.15.99:0/3483503704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c18468f0 0x5565c1851410 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54be7fc640 0 client.15405 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 >> 172.21.15.99:0/3483503704 conn(0x5565c1842e00 msgr2=0x5565c1842100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 shutdown_connections 2024-05-14T17:04:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 1 -- 172.21.15.99:0/3483503704 wait complete. 2024-05-14T17:04:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.643+0000 7f54e0413740 20 client.15405 trim_cache size 0 max 16384 2024-05-14T17:04:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 Processor -- start 2024-05-14T17:04:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 -- start start 2024-05-14T17:04:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1841f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1850f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1869110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:48.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81b30b0 con 0x5565c1851ad0 2024-05-14T17:04:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81b0980 con 0x5565c1847210 2024-05-14T17:04:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54e0413740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81ae7d0 con 0x5565c18468f0 2024-05-14T17:04:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1869110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1869110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38664/0 (socket says 172.21.15.99:38664) 2024-05-14T17:04:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.658+0000 7f54d77fe640 1 -- 172.21.15.99:0/2779771185 learned_addr learned my addr 172.21.15.99:0/2779771185 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54cffff640 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1850f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1841f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d77fe640 1 -- 172.21.15.99:0/2779771185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 msgr2=0x5565c1841f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d77fe640 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1841f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d77fe640 1 -- 172.21.15.99:0/2779771185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 msgr2=0x5565c1850f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d77fe640 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1850f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d77fe640 1 -- 172.21.15.99:0/2779771185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c1863e80 con 0x5565c1851ad0 2024-05-14T17:04:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1841f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54d77fe640 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1869110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f54d000f750 tx=0x7f54d0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91df2f4d5fe64e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54ccff9640 1 -- 172.21.15.99:0/2779771185 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d002a070 con 0x5565c1851ad0 2024-05-14T17:04:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54ccff9640 1 -- 172.21.15.99:0/2779771185 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d001a990 con 0x5565c1851ad0 2024-05-14T17:04:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.659+0000 7f54ccff9640 1 -- 172.21.15.99:0/2779771185 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00258c0 con 0x5565c1851ad0 2024-05-14T17:04:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 -- 172.21.15.99:0/2779771185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 msgr2=0x5565c1869110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1869110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f54d000f750 tx=0x7f54d0014040 comp rx=0 tx=0).stop 2024-05-14T17:04:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 -- 172.21.15.99:0/2779771185 shutdown_connections 2024-05-14T17:04:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1869110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1850f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 --2- 172.21.15.99:0/2779771185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1841f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 -- 172.21.15.99:0/2779771185 >> 172.21.15.99:0/2779771185 conn(0x5565c1842e00 msgr2=0x5565c1841870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 -- 172.21.15.99:0/2779771185 shutdown_connections 2024-05-14T17:04:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 -- 172.21.15.99:0/2779771185 wait complete. 2024-05-14T17:04:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 -- start start 2024-05-14T17:04:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 Processor -- start 2024-05-14T17:04:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.660+0000 7f54e0413740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1855590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1855ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1856010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81aee40 con 0x5565c1851ad0 2024-05-14T17:04:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81b02b0 con 0x5565c1847210 2024-05-14T17:04:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81af510 con 0x5565c18468f0 2024-05-14T17:04:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 10 client.0 _ll_register_callbacks cb 0x7f54c8106ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1856010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1855590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1856010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38680/0 (socket says 172.21.15.99:38680) 2024-05-14T17:04:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 -- 172.21.15.99:0/675042368 learned_addr learned my addr 172.21.15.99:0/675042368 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54cffff640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1855ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:49.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 -- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 msgr2=0x5565c1855590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1855590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 -- 172.21.15.99:0/675042368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 msgr2=0x5565c1855ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1855ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 -- 172.21.15.99:0/675042368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c1842c30 con 0x5565c1851ad0 2024-05-14T17:04:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54cffff640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1855ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54d77fe640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1856010 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f54d001a7c0 tx=0x7f54d001a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ac5376fc89d48f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:04:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54be7fc640 1 -- 172.21.15.99:0/675042368 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0033060 con 0x5565c1851ad0 2024-05-14T17:04:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54be7fc640 1 -- 172.21.15.99:0/675042368 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f54d0017040 con 0x5565c1851ad0 2024-05-14T17:04:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d001a990 con 0x5565c1851ad0 2024-05-14T17:04:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54be7fc640 1 -- 172.21.15.99:0/675042368 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54d002a070 con 0x5565c1851ad0 2024-05-14T17:04:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 10 client.15411 fetch_fsmap learned FSMap version 26 2024-05-14T17:04:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.661+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565c18cc0a0 con 0x5565c1851ad0 2024-05-14T17:04:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54be7fc640 1 -- 172.21.15.99:0/675042368 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d001a990 con 0x5565c1851ad0 2024-05-14T17:04:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54be7fc640 1 -- 172.21.15.99:0/675042368 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f54d0025a00 con 0x5565c1851ad0 2024-05-14T17:04:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54e0413740 10 client.15411 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:04:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54e0413740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:04:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54e0413740 20 client.15411 populate_metadata read hostname 'smithi099' 2024-05-14T17:04:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54e0413740 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c18cc790 0x5565c18cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54e0413740 4 client.15411 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565c17b5360 con 0x5565c18cc790 2024-05-14T17:04:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.662+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c18cc790 0x5565c18cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.663+0000 7f54be7fc640 10 client.15411 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.663+0000 7f54d6ffd640 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c18cc790 0x5565c18cebb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0124c8b2278f928 server_cookie=6b907a440655da3c in_seq=0 out_seq=0 2024-05-14T17:04:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54be7fc640 1 -- 172.21.15.99:0/675042368 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5565c17b5360 con 0x5565c18cc790 2024-05-14T17:04:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54be7fc640 10 client.15411 handle_command_reply: tid=0 2024-05-14T17:04:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 client.15411 shutdown 2024-05-14T17:04:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 msgr2=0x5565c1856010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1856010 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f54d001a7c0 tx=0x7f54d001a7f0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 shutdown_connections 2024-05-14T17:04:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5565c18cc790 0x5565c18cebb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c1851ad0 0x5565c1856010 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54be7fc640 0 client.15411 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:04:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565c1847210 0x5565c1855ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 --2- 172.21.15.99:0/675042368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565c18468f0 0x5565c1855590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 >> 172.21.15.99:0/675042368 conn(0x5565c1842e00 msgr2=0x5565c1842100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 shutdown_connections 2024-05-14T17:04:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 1 -- 172.21.15.99:0/675042368 wait complete. 2024-05-14T17:04:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.666+0000 7f54e0413740 20 client.15411 trim_cache size 0 max 16384 2024-05-14T17:04:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:04:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:04:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:04:49.968 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:48 smithi193 ceph-mon[35962]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:04:49.976 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:48 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:49.983 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:48 smithi150 ceph-mon[36204]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:04:49.990 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:48 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f54c40b2b30 msgr2=0x7f54c40b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f54c40b2b30 0x7f54c40b4ff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f54d806bcf0 tx=0x7f54d0004a40 comp rx=0 tx=0).stop 2024-05-14T17:04:50.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 msgr2=0x7f54d819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d819b3d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005940 tx=0x7f54c8004270 comp rx=0 tx=0).stop 2024-05-14T17:04:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 shutdown_connections 2024-05-14T17:04:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f54c40b2b30 0x7f54c40b4ff0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d810d0b0 0x7f54d819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54d8104600 0x7f54d819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 --2- 172.21.15.99:0/3314748243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d8103ce0 0x7f54d819b3d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 >> 172.21.15.99:0/3314748243 conn(0x7f54d8078a60 msgr2=0x7f54d8108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 shutdown_connections 2024-05-14T17:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:46.667+0000 7f54dd6fc640 1 -- 172.21.15.99:0/3314748243 wait complete. 2024-05-14T17:04:50.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:46 smithi099 ceph-mon[31851]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:04:50.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:48 smithi099 ceph-mon[31851]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:04:50.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:48 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:50.255 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:50.262 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:04:50.391 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:50.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:50.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:04:50.440 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:50.449 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:50.458 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:04:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:50 smithi150 ceph-mon[36204]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-14T17:04:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:50 smithi099 ceph-mon[31851]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-14T17:04:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:50 smithi193 ceph-mon[35962]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-14T17:04:53.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:52 smithi150 ceph-mon[36204]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-14T17:04:53.278 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:52 smithi099 ceph-mon[31851]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-14T17:04:53.364 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:52 smithi193 ceph-mon[35962]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-14T17:04:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:54 smithi150 ceph-mon[36204]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-14T17:04:55.238 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.2 2024-05-14T17:04:55.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:54 smithi099 ceph-mon[31851]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-14T17:04:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:54 smithi193 ceph-mon[35962]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-14T17:04:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:56.936 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:56 smithi099 ceph-mon[31851]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-05-14T17:04:57.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:56 smithi150 ceph-mon[36204]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-05-14T17:04:57.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 -- 172.21.15.99:0/2086684668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102440 msgr2=0x7f58c41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:57.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/2086684668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102440 0x7f58c41028c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009990 tx=0x7f58b4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 -- 172.21.15.99:0/2086684668 shutdown_connections 2024-05-14T17:04:57.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/2086684668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 0x7f58c410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/2086684668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4108520 0x7f58c4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/2086684668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102440 0x7f58c41028c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 -- 172.21.15.99:0/2086684668 >> 172.21.15.99:0/2086684668 conn(0x7f58c40793f0 msgr2=0x7f58c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:57.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 -- 172.21.15.99:0/2086684668 shutdown_connections 2024-05-14T17:04:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.269+0000 7f58c8fb9700 1 -- 172.21.15.99:0/2086684668 wait complete. 2024-05-14T17:04:57.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:56 smithi193 ceph-mon[35962]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-05-14T17:04:57.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.270+0000 7f58c8fb9700 1 Processor -- start 2024-05-14T17:04:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.270+0000 7f58c8fb9700 1 -- start start 2024-05-14T17:04:57.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c8fb9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 0x7f58c419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c8fb9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 0x7f58c419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c8fb9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4108520 0x7f58c4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c8fb9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41125f0 con 0x7f58c4108520 2024-05-14T17:04:57.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c8fb9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4112470 con 0x7f58c4102440 2024-05-14T17:04:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c8fb9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4112770 con 0x7f58c4102e00 2024-05-14T17:04:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 0x7f58c419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:57.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c1d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 0x7f58c419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:57.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c1d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 0x7f58c419fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:59556/0 (socket says 172.21.15.99:59556) 2024-05-14T17:04:57.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 0x7f58c419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:36226/0 (socket says 172.21.15.99:36226) 2024-05-14T17:04:57.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c1d9b700 1 -- 172.21.15.99:0/692322536 learned_addr learned my addr 172.21.15.99:0/692322536 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:04:57.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c2d9d700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4108520 0x7f58c4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:57.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 -- 172.21.15.99:0/692322536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 msgr2=0x7f58c419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:57.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 0x7f58c419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 -- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4108520 msgr2=0x7f58c4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:57.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4108520 0x7f58c4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:57.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 -- 172.21.15.99:0/692322536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c419a1c0 con 0x7f58c4102440 2024-05-14T17:04:57.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c2d9d700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4108520 0x7f58c4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:57.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.271+0000 7f58c259c700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 0x7f58c419f8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d950 tx=0x7f58ac00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:57.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.272+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac016070 con 0x7f58c4102440 2024-05-14T17:04:57.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.272+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58ac00fd40 con 0x7f58c4102440 2024-05-14T17:04:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.272+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c419a4b0 con 0x7f58c4102440 2024-05-14T17:04:57.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.272+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00e3b0 con 0x7f58c4102440 2024-05-14T17:04:57.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.272+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c41a6770 con 0x7f58c4102440 2024-05-14T17:04:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.272+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c406a230 con 0x7f58c4102440 2024-05-14T17:04:57.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.274+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f58ac019070 con 0x7f58c4102440 2024-05-14T17:04:57.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.275+0000 7f58bb7fe700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58b00b2b10 0x7f58b00b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:57.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.275+0000 7f58c1d9b700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58b00b2b10 0x7f58b00b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:57.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.275+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f58ac0e2730 con 0x7f58c4102440 2024-05-14T17:04:57.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.275+0000 7f58c1d9b700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58b00b2b10 0x7f58b00b4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58b4003800 tx=0x7f58b4005bc0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:57.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.279+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58ac0a9ef0 con 0x7f58c4102440 2024-05-14T17:04:57.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.533+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c419af90 con 0x7f58c4102440 2024-05-14T17:04:57.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.534+0000 7f58bb7fe700 1 -- 172.21.15.99:0/692322536 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f58ac0a97f0 con 0x7f58c4102440 2024-05-14T17:04:58.055 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:04:58.063 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:04:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58b00b2b10 msgr2=0x7f58b00b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58b00b2b10 0x7f58b00b4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58b4003800 tx=0x7f58b4005bc0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 msgr2=0x7f58c419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:58.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 0x7f58c419f8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d950 tx=0x7f58ac00dc60 comp rx=0 tx=0).stop 2024-05-14T17:04:58.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 shutdown_connections 2024-05-14T17:04:58.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58b00b2b10 0x7f58b00b4fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4108520 0x7f58c4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c4102440 0x7f58c419f8a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 --2- 172.21.15.99:0/692322536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58c4102e00 0x7f58c419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:58.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 >> 172.21.15.99:0/692322536 conn(0x7f58c40793f0 msgr2=0x7f58c4106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:58.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 shutdown_connections 2024-05-14T17:04:58.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:04:57.536+0000 7f58c8fb9700 1 -- 172.21.15.99:0/692322536 wait complete. 2024-05-14T17:04:58.216 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:04:58.227 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:57 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/692322536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:58.246 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:57 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/692322536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:58.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:57 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/692322536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:58.319 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 135.2 --await-for 45.0' 2024-05-14T17:04:58.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 135.2 --await-for 45.0 2024-05-14T17:04:58.604 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:58.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:04:58 smithi099 ceph-mon[31851]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-05-14T17:04:59.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:04:58 smithi150 ceph-mon[36204]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-05-14T17:04:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:04:58 smithi193 ceph-mon[35962]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-05-14T17:04:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 -- 172.21.15.99:0/1951384325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e241025f0 msgr2=0x7f3e2410e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 --2- 172.21.15.99:0/1951384325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e241025f0 0x7f3e2410e7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009be0 tx=0x7f3e1c02a6c0 comp rx=0 tx=0).stop 2024-05-14T17:04:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 -- 172.21.15.99:0/1951384325 shutdown_connections 2024-05-14T17:04:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 --2- 172.21.15.99:0/1951384325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e241025f0 0x7f3e2410e7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 --2- 172.21.15.99:0/1951384325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 0x7f3e241020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 --2- 172.21.15.99:0/1951384325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e24106920 0x7f3e24106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 -- 172.21.15.99:0/1951384325 >> 172.21.15.99:0/1951384325 conn(0x7f3e24078730 msgr2=0x7f3e24078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 -- 172.21.15.99:0/1951384325 shutdown_connections 2024-05-14T17:04:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.758+0000 7f3e297c9640 1 -- 172.21.15.99:0/1951384325 wait complete. 2024-05-14T17:04:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 Processor -- start 2024-05-14T17:04:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 -- start start 2024-05-14T17:04:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 0x7f3e24199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e241025f0 0x7f3e24199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 0x7f3e2419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24111bc0 con 0x7f3e24106920 2024-05-14T17:04:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24111a40 con 0x7f3e24101c30 2024-05-14T17:05:00.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e297c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24111d40 con 0x7f3e241025f0 2024-05-14T17:05:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 0x7f3e2419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 0x7f3e2419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34066/0 (socket says 172.21.15.99:34066) 2024-05-14T17:05:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e237fe640 1 -- 172.21.15.99:0/2232283320 learned_addr learned my addr 172.21.15.99:0/2232283320 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e227fc640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e241025f0 0x7f3e24199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.759+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 0x7f3e24199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e237fe640 1 -- 172.21.15.99:0/2232283320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e241025f0 msgr2=0x7f3e24199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e237fe640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e241025f0 0x7f3e24199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e237fe640 1 -- 172.21.15.99:0/2232283320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 msgr2=0x7f3e24199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e237fe640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 0x7f3e24199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e237fe640 1 -- 172.21.15.99:0/2232283320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419e030 con 0x7f3e24106920 2024-05-14T17:05:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 0x7f3e24199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e237fe640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 0x7f3e2419d900 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c004af0 tx=0x7f3e1c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e1bfff640 1 -- 172.21.15.99:0/2232283320 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c041020 con 0x7f3e24106920 2024-05-14T17:05:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419e2c0 con 0x7f3e24106920 2024-05-14T17:05:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e1bfff640 1 -- 172.21.15.99:0/2232283320 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1c003e10 con 0x7f3e24106920 2024-05-14T17:05:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e1bfff640 1 -- 172.21.15.99:0/2232283320 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c036040 con 0x7f3e24106920 2024-05-14T17:05:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.760+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241a5bb0 con 0x7f3e24106920 2024-05-14T17:05:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.762+0000 7f3e1bfff640 1 -- 172.21.15.99:0/2232283320 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f3e1c044020 con 0x7f3e24106920 2024-05-14T17:05:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.762+0000 7f3e2c5bd740 1 Processor -- start 2024-05-14T17:05:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 -- start start 2024-05-14T17:05:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x56095089fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560950812c30 con 0x5609508aaad0 2024-05-14T17:05:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609508027d0 con 0x5609508a0210 2024-05-14T17:05:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560950806810 con 0x56095089f8f0 2024-05-14T17:05:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e227fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e227fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34068/0 (socket says 172.21.15.99:34068) 2024-05-14T17:05:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e227fc640 1 -- 172.21.15.99:0/879254942 learned_addr learned my addr 172.21.15.99:0/879254942 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e227fc640 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x56095089fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.763+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e227fc640 1 -- 172.21.15.99:0/879254942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 msgr2=0x56095089fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e227fc640 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x56095089fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e227fc640 1 -- 172.21.15.99:0/879254942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 msgr2=0x5609508aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e227fc640 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e227fc640 1 -- 172.21.15.99:0/879254942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560950517210 con 0x5609508aaad0 2024-05-14T17:05:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e1bfff640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3e100b2b30 0x7f3e100b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3e100b2b30 0x7f3e100b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e227fc640 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508acec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c012780 tx=0x7f3e0c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef44621127f6d37a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e1bfff640 1 -- 172.21.15.99:0/2232283320 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3e1c103430 con 0x7f3e24106920 2024-05-14T17:05:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e197fa640 1 -- 172.21.15.99:0/879254942 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c012d00 con 0x5609508aaad0 2024-05-14T17:05:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e197fa640 1 -- 172.21.15.99:0/879254942 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c0107d0 con 0x5609508aaad0 2024-05-14T17:05:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.764+0000 7f3e197fa640 1 -- 172.21.15.99:0/879254942 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c003850 con 0x5609508aaad0 2024-05-14T17:05:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3e100b2b30 0x7f3e100b4ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5609508a0e90 tx=0x7f3e14005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/879254942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 msgr2=0x5609508acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508acec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c012780 tx=0x7f3e0c0096e0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/879254942 shutdown_connections 2024-05-14T17:05:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508acec0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/879254942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x56095089fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/879254942 >> 172.21.15.99:0/879254942 conn(0x56095089be00 msgr2=0x56095089c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/879254942 shutdown_connections 2024-05-14T17:05:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.765+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/879254942 wait complete. 2024-05-14T17:05:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 -- start start 2024-05-14T17:05:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 Processor -- start 2024-05-14T17:05:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609504e8790 con 0x5609508aaad0 2024-05-14T17:05:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609507be890 con 0x5609508a0210 2024-05-14T17:05:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5609508ad7a0 con 0x56095089f8f0 2024-05-14T17:05:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e2c5bd740 10 client.0 _ll_register_callbacks cb 0x5609507c1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508a7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34076/0 (socket says 172.21.15.99:34076) 2024-05-14T17:05:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e237fe640 1 -- 172.21.15.99:0/4032284340 learned_addr learned my addr 172.21.15.99:0/4032284340 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e227fc640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.766+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e237fe640 1 -- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 msgr2=0x5609508aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e237fe640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e237fe640 1 -- 172.21.15.99:0/4032284340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 msgr2=0x5609508a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e237fe640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508a77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e237fe640 1 -- 172.21.15.99:0/4032284340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609505e3620 con 0x5609508aaad0 2024-05-14T17:05:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e237fe640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508a7dd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0ca2c0 tx=0x7f3e1c0cabe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f3d0fa1a6c0a2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/4032284340 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c0cacc0 con 0x5609508aaad0 2024-05-14T17:05:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/4032284340 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3e1c100d10 con 0x5609508aaad0 2024-05-14T17:05:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c0107d0 con 0x5609508aaad0 2024-05-14T17:05:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/4032284340 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c030970 con 0x5609508aaad0 2024-05-14T17:05:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e2c5bd740 10 client.15429 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609508b9280 con 0x5609508aaad0 2024-05-14T17:05:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/4032284340 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c0107d0 con 0x5609508aaad0 2024-05-14T17:05:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/4032284340 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3e1c0c4ce0 con 0x5609508aaad0 2024-05-14T17:05:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e2c5bd740 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.767+0000 7f3e2c5bd740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e2c5bd740 20 client.15429 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5609508ba780 0x5609508bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e2c5bd740 4 client.15429 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:01.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560950814b80 con 0x5609508ba780 2024-05-14T17:05:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5609508ba780 0x5609508bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5609508ba780 0x5609508bcb70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d8f17aa84a8854 server_cookie=e94b393fee0e0df0 in_seq=0 out_seq=0 2024-05-14T17:05:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.768+0000 7f3e0a7fc640 10 client.15429 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/4032284340 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560950814b80 con 0x5609508ba780 2024-05-14T17:05:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e0a7fc640 10 client.15429 handle_command_reply: tid=0 2024-05-14T17:05:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 client.15429 shutdown 2024-05-14T17:05:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 msgr2=0x5609508a7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508a7dd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0ca2c0 tx=0x7f3e1c0cabe0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 shutdown_connections 2024-05-14T17:05:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5609508ba780 0x5609508bcb70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508a7dd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508a0210 0x5609508a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e0a7fc640 0 client.15429 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/4032284340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 >> 172.21.15.99:0/4032284340 conn(0x56095089be00 msgr2=0x56095089b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 shutdown_connections 2024-05-14T17:05:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/4032284340 wait complete. 2024-05-14T17:05:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.769+0000 7f3e2c5bd740 20 client.15429 trim_cache size 0 max 16384 2024-05-14T17:05:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 Processor -- start 2024-05-14T17:05:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 -- start start 2024-05-14T17:05:01.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 0x5609508aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508aaad0 0x5609508c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241b2f00 con 0x5609508a0210 2024-05-14T17:05:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241b07d0 con 0x5609508aaad0 2024-05-14T17:05:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.790+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241ae620 con 0x56095089f8f0 2024-05-14T17:05:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 0x5609508aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 0x5609508aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34086/0 (socket says 172.21.15.99:34086) 2024-05-14T17:05:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 -- 172.21.15.99:0/3427444815 learned_addr learned my addr 172.21.15.99:0/3427444815 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e237fe640 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508aaad0 0x5609508c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 -- 172.21.15.99:0/3427444815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 msgr2=0x5609508a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 -- 172.21.15.99:0/3427444815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508aaad0 msgr2=0x5609508c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508aaad0 0x5609508c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 -- 172.21.15.99:0/3427444815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609508a2150 con 0x5609508a0210 2024-05-14T17:05:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:02.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e227fc640 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 0x5609508aa000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c012720 tx=0x7f3e0c012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fde902f1a206bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e18ff9640 1 -- 172.21.15.99:0/3427444815 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c027070 con 0x5609508a0210 2024-05-14T17:05:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e18ff9640 1 -- 172.21.15.99:0/3427444815 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c016040 con 0x5609508a0210 2024-05-14T17:05:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.791+0000 7f3e18ff9640 1 -- 172.21.15.99:0/3427444815 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c010e20 con 0x5609508a0210 2024-05-14T17:05:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3427444815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 msgr2=0x5609508aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 0x5609508aa000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c012720 tx=0x7f3e0c012e90 comp rx=0 tx=0).stop 2024-05-14T17:05:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3427444815 shutdown_connections 2024-05-14T17:05:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609508aaad0 0x5609508c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508a0210 0x5609508aa000 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3427444815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095089f8f0 0x5609508a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3427444815 >> 172.21.15.99:0/3427444815 conn(0x56095089be00 msgr2=0x5609508a2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3427444815 shutdown_connections 2024-05-14T17:05:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.792+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3427444815 wait complete. 2024-05-14T17:05:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 -- start start 2024-05-14T17:05:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 Processor -- start 2024-05-14T17:05:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56095089f8f0 0x5609508b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609508a0210 0x5609508b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241aec90 con 0x5609508aaad0 2024-05-14T17:05:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241b0100 con 0x56095089f8f0 2024-05-14T17:05:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241af360 con 0x5609508a0210 2024-05-14T17:05:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e2c5bd740 10 client.0 _ll_register_callbacks cb 0x7f3e1c0c9f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34096/0 (socket says 172.21.15.99:34096) 2024-05-14T17:05:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 -- 172.21.15.99:0/3696570474 learned_addr learned my addr 172.21.15.99:0/3696570474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56095089f8f0 0x5609508b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e227fc640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609508a0210 0x5609508b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 -- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609508a0210 msgr2=0x5609508b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609508a0210 0x5609508b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 -- 172.21.15.99:0/3696570474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56095089f8f0 msgr2=0x5609508b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56095089f8f0 0x5609508b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.793+0000 7f3e237fe640 1 -- 172.21.15.99:0/3696570474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609508a6790 con 0x5609508aaad0 2024-05-14T17:05:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e237fe640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508af9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0ca550 tx=0x7f3e1c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9583ec6bba4c5d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/3696570474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c004e40 con 0x5609508aaad0 2024-05-14T17:05:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/3696570474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f3e1c0c6140 con 0x5609508aaad0 2024-05-14T17:05:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c016040 con 0x5609508aaad0 2024-05-14T17:05:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/3696570474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c0cacc0 con 0x5609508aaad0 2024-05-14T17:05:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 10 client.15435 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609508bcee0 con 0x5609508aaad0 2024-05-14T17:05:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/3696570474 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0c016040 con 0x5609508aaad0 2024-05-14T17:05:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/3696570474 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3e1c0c4ce0 con 0x5609508aaad0 2024-05-14T17:05:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 20 client.15435 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560950925b30 0x560950927f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 4 client.15435 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.794+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56095080e360 con 0x560950925b30 2024-05-14T17:05:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.795+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560950925b30 0x560950927f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.795+0000 7f3e0a7fc640 10 client.15435 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.795+0000 7f3e22ffd640 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560950925b30 0x560950927f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d98e2ec4a79c40 server_cookie=3c33ed71f984ba01 in_seq=0 out_seq=0 2024-05-14T17:05:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e0a7fc640 1 -- 172.21.15.99:0/3696570474 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56095080e360 con 0x560950925b30 2024-05-14T17:05:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e0a7fc640 10 client.15435 handle_command_reply: tid=0 2024-05-14T17:05:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 client.15435 shutdown 2024-05-14T17:05:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 msgr2=0x5609508af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508af9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0ca550 tx=0x7f3e1c004d50 comp rx=0 tx=0).stop 2024-05-14T17:05:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 shutdown_connections 2024-05-14T17:05:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560950925b30 0x560950927f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609508aaad0 0x5609508af9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609508a0210 0x5609508b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 --2- 172.21.15.99:0/3696570474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56095089f8f0 0x5609508b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e0a7fc640 0 client.15435 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 >> 172.21.15.99:0/3696570474 conn(0x56095089be00 msgr2=0x56095089a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 shutdown_connections 2024-05-14T17:05:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 1 -- 172.21.15.99:0/3696570474 wait complete. 2024-05-14T17:05:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.802+0000 7f3e2c5bd740 20 client.15435 trim_cache size 0 max 16384 2024-05-14T17:05:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:05:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:05:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:05:03.513 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:00 smithi193 ceph-mon[35962]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-14T17:05:03.522 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:02 smithi193 ceph-mon[35962]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T17:05:03.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:00 smithi150 ceph-mon[36204]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-14T17:05:03.544 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:02 smithi150 ceph-mon[36204]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T17:05:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3e100b2b30 msgr2=0x7f3e100b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3e100b2b30 0x7f3e100b4ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5609508a0e90 tx=0x7f3e14005f90 comp rx=0 tx=0).stop 2024-05-14T17:05:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 msgr2=0x7f3e2419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 0x7f3e2419d900 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c004af0 tx=0x7f3e1c003c20 comp rx=0 tx=0).stop 2024-05-14T17:05:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 shutdown_connections 2024-05-14T17:05:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3e100b2b30 0x7f3e100b4ff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e24106920 0x7f3e2419d900 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e241025f0 0x7f3e24199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 --2- 172.21.15.99:0/2232283320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e24101c30 0x7f3e24199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 >> 172.21.15.99:0/2232283320 conn(0x7f3e24078730 msgr2=0x7f3e2406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 shutdown_connections 2024-05-14T17:05:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:04:59.803+0000 7f3e297c9640 1 -- 172.21.15.99:0/2232283320 wait complete. 2024-05-14T17:05:03.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 -- 172.21.15.99:0/3658213325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c109630 msgr2=0x7efc5c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:03.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 --2- 172.21.15.99:0/3658213325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c109630 0x7efc5c109a10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0099c0 tx=0x7efc4c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 -- 172.21.15.99:0/3658213325 shutdown_connections 2024-05-14T17:05:03.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 --2- 172.21.15.99:0/3658213325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c103f80 0x7efc5c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 --2- 172.21.15.99:0/3658213325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c109630 0x7efc5c109a10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 --2- 172.21.15.99:0/3658213325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c1035c0 0x7efc5c103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:03.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.252+0000 7efc61faf700 1 -- 172.21.15.99:0/3658213325 >> 172.21.15.99:0/3658213325 conn(0x7efc5c07a660 msgr2=0x7efc5c07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:03.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.253+0000 7efc61faf700 1 -- 172.21.15.99:0/3658213325 shutdown_connections 2024-05-14T17:05:03.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.253+0000 7efc61faf700 1 -- 172.21.15.99:0/3658213325 wait complete. 2024-05-14T17:05:03.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.253+0000 7efc61faf700 1 Processor -- start 2024-05-14T17:05:03.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.253+0000 7efc61faf700 1 -- start start 2024-05-14T17:05:03.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc61faf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 0x7efc5c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:03.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc61faf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c103f80 0x7efc5c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:03.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc61faf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c109630 0x7efc5c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc61faf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c113750 con 0x7efc5c1035c0 2024-05-14T17:05:04.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc61faf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1135d0 con 0x7efc5c109630 2024-05-14T17:05:04.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc61faf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1138d0 con 0x7efc5c103f80 2024-05-14T17:05:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 0x7efc5c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 0x7efc5c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34112/0 (socket says 172.21.15.99:34112) 2024-05-14T17:05:04.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5b7fe700 1 -- 172.21.15.99:0/2254742347 learned_addr learned my addr 172.21.15.99:0/2254742347 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:04.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5affd700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c103f80 0x7efc5c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5bfff700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c109630 0x7efc5c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5affd700 1 -- 172.21.15.99:0/2254742347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c109630 msgr2=0x7efc5c1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5affd700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c109630 0x7efc5c1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5affd700 1 -- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 msgr2=0x7efc5c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5affd700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 0x7efc5c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5affd700 1 -- 172.21.15.99:0/2254742347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c19a050 con 0x7efc5c103f80 2024-05-14T17:05:04.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.254+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 0x7efc5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:04.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.255+0000 7efc5affd700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c103f80 0x7efc5c19fc20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e9f0 tx=0x7efc5000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:04.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.255+0000 7efc58ff9700 1 -- 172.21.15.99:0/2254742347 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000cbd0 con 0x7efc5c103f80 2024-05-14T17:05:04.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.255+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c19a340 con 0x7efc5c103f80 2024-05-14T17:05:04.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.255+0000 7efc58ff9700 1 -- 172.21.15.99:0/2254742347 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc500044d0 con 0x7efc5c103f80 2024-05-14T17:05:04.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.255+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c1a65b0 con 0x7efc5c103f80 2024-05-14T17:05:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.255+0000 7efc58ff9700 1 -- 172.21.15.99:0/2254742347 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50005020 con 0x7efc5c103f80 2024-05-14T17:05:04.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.258+0000 7efc58ff9700 1 -- 172.21.15.99:0/2254742347 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7efc500052a0 con 0x7efc5c103f80 2024-05-14T17:05:04.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.259+0000 7efc58ff9700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efc440b2c40 0x7efc440b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.259+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efc440b2c40 0x7efc440b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.259+0000 7efc58ff9700 1 -- 172.21.15.99:0/2254742347 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efc500e2700 con 0x7efc5c103f80 2024-05-14T17:05:04.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.259+0000 7efc6c845b80 1 Processor -- start 2024-05-14T17:05:04.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.259+0000 7efc6c845b80 1 -- start start 2024-05-14T17:05:04.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef98c0 0x55613fef9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613fef5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fef5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc6c845b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55613fe4ef00 con 0x55613fef5760 2024-05-14T17:05:04.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efc440b2c40 0x7efc440b5100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc4c003760 tx=0x7efc4c005ad0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc6c845b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55613fe3eaa0 con 0x55613fef98c0 2024-05-14T17:05:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc6c845b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55613fe42ae0 con 0x55613fefa1e0 2024-05-14T17:05:04.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fef5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613fef5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fef5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34118/0 (socket says 172.21.15.99:34118) 2024-05-14T17:05:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 -- 172.21.15.99:0/2248936603 learned_addr learned my addr 172.21.15.99:0/2248936603 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:04.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef98c0 0x55613fef9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 -- 172.21.15.99:0/2248936603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 msgr2=0x55613fef5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:04.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613fef5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 -- 172.21.15.99:0/2248936603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef98c0 msgr2=0x55613fef9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:04.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef98c0 0x55613fef9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 -- 172.21.15.99:0/2248936603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613fde8140 con 0x55613fef5760 2024-05-14T17:05:04.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc5bfff700 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fef5bc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc5400e190 tx=0x7efc5400e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ba120ed70a4cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:04.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.260+0000 7efc41ffb700 1 -- 172.21.15.99:0/2248936603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5400ed30 con 0x55613fef5760 2024-05-14T17:05:04.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.261+0000 7efc41ffb700 1 -- 172.21.15.99:0/2248936603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc54004bb0 con 0x55613fef5760 2024-05-14T17:05:04.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.261+0000 7efc41ffb700 1 -- 172.21.15.99:0/2248936603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54014dd0 con 0x55613fef5760 2024-05-14T17:05:04.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.261+0000 7efc6c845b80 1 -- 172.21.15.99:0/2248936603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 msgr2=0x55613fef5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.261+0000 7efc6c845b80 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fef5bc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc5400e190 tx=0x7efc5400e4a0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 -- 172.21.15.99:0/2248936603 shutdown_connections 2024-05-14T17:05:04.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fef5bc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef98c0 0x55613fef9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 --2- 172.21.15.99:0/2248936603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613fef5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 -- 172.21.15.99:0/2248936603 >> 172.21.15.99:0/2248936603 conn(0x55613fe1df40 msgr2=0x55613feeb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:04.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 -- 172.21.15.99:0/2248936603 shutdown_connections 2024-05-14T17:05:04.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 -- 172.21.15.99:0/2248936603 wait complete. 2024-05-14T17:05:04.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 -- start start 2024-05-14T17:05:05.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.262+0000 7efc6c845b80 1 Processor -- start 2024-05-14T17:05:05.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef5760 0x55613fef4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613fef4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613feedd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55613fd45130 con 0x55613fef98c0 2024-05-14T17:05:05.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55613fda6f30 con 0x55613fef5760 2024-05-14T17:05:05.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55613ff01d70 con 0x55613fefa1e0 2024-05-14T17:05:05.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613fef4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613fef4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34120/0 (socket says 172.21.15.99:34120) 2024-05-14T17:05:05.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5affd700 1 -- 172.21.15.99:0/504613017 learned_addr learned my addr 172.21.15.99:0/504613017 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:05.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc6c845b80 10 client.0 _ll_register_callbacks cb 0x55613fdd9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:05.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef5760 0x55613fef4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5b7fe700 1 -- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 msgr2=0x55613feedd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:05:05.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613feedd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5b7fe700 1 -- 172.21.15.99:0/504613017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 msgr2=0x55613fef4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:05.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613fef4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5b7fe700 1 -- 172.21.15.99:0/504613017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613fd8ea60 con 0x55613fef5760 2024-05-14T17:05:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.263+0000 7efc5affd700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613fef4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:05.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef5760 0x55613fef4900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009990 tx=0x7efc4c01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ef7d0f8239d2ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:05.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc3affd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:05:05.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc3affd700 1 -- 172.21.15.99:0/504613017 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c024820 con 0x55613fef5760 2024-05-14T17:05:05.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc54004bb0 con 0x55613fef5760 2024-05-14T17:05:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc3affd700 1 -- 172.21.15.99:0/504613017 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efc4c0249c0 con 0x55613fef5760 2024-05-14T17:05:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc3affd700 1 -- 172.21.15.99:0/504613017 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc4c024bc0 con 0x55613fef5760 2024-05-14T17:05:05.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc6c845b80 10 client.24926 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:05.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55613fdf88f0 con 0x55613fef5760 2024-05-14T17:05:05.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.264+0000 7efc3affd700 1 -- 172.21.15.99:0/504613017 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc54004bb0 con 0x55613fef5760 2024-05-14T17:05:05.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc3affd700 1 -- 172.21.15.99:0/504613017 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efc4c030c80 con 0x55613fef5760 2024-05-14T17:05:05.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc6c845b80 10 client.24926 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:05.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc6c845b80 10 client.24926 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:05:05.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc6c845b80 20 client.24926 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:05.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc6c845b80 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff12c40 0x55613ff15070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc6c845b80 4 client.24926 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:05.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55613fe50e50 con 0x55613ff12c40 2024-05-14T17:05:05.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.265+0000 7efc5affd700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff12c40 0x55613ff15070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc3affd700 10 client.24926 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:05.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc5affd700 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff12c40 0x55613ff15070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe704358ff052fe server_cookie=f63fe0135e248edd in_seq=0 out_seq=0 2024-05-14T17:05:05.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc3affd700 1 -- 172.21.15.99:0/504613017 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55613fe50e50 con 0x55613ff12c40 2024-05-14T17:05:05.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc3affd700 10 client.24926 handle_command_reply: tid=0 2024-05-14T17:05:05.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc6c845b80 1 client.24926 shutdown 2024-05-14T17:05:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef5760 msgr2=0x55613fef4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:05.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.266+0000 7efc6c845b80 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef5760 0x55613fef4900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009990 tx=0x7efc4c01a050 comp rx=0 tx=0).stop 2024-05-14T17:05:05.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 shutdown_connections 2024-05-14T17:05:05.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fefa1e0 0x55613feedd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fef5760 0x55613fef4900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff12c40 0x55613ff15070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 --2- 172.21.15.99:0/504613017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613fef4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 >> 172.21.15.99:0/504613017 conn(0x55613fe1df40 msgr2=0x55613fee6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc3affd700 0 client.24926 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:05.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 shutdown_connections 2024-05-14T17:05:06.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 1 -- 172.21.15.99:0/504613017 wait complete. 2024-05-14T17:05:06.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.267+0000 7efc6c845b80 20 client.24926 trim_cache size 0 max 16384 2024-05-14T17:05:06.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 Processor -- start 2024-05-14T17:05:06.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 -- start start 2024-05-14T17:05:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fee6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef98c0 0x55613fefb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613fefbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1ae300 con 0x55613fef5760 2024-05-14T17:05:06.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1abb90 con 0x55613fefa1e0 2024-05-14T17:05:06.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.289+0000 7efc6c845b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1a9a00 con 0x55613fef98c0 2024-05-14T17:05:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fee6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fee6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34132/0 (socket says 172.21.15.99:34132) 2024-05-14T17:05:06.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5b7fe700 1 -- 172.21.15.99:0/1681226596 learned_addr learned my addr 172.21.15.99:0/1681226596 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:06.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef98c0 0x55613fefb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5bfff700 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613fefbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 -- 172.21.15.99:0/1681226596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 msgr2=0x55613fefbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613fefbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 -- 172.21.15.99:0/1681226596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 msgr2=0x55613fee6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fee6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 -- 172.21.15.99:0/1681226596 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613fef32f0 con 0x55613fef98c0 2024-05-14T17:05:06.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fee6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc5affd700 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef98c0 0x55613fefb6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c5b0 tx=0x7efc500cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e382bf93ae93ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:06.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc417fa700 1 -- 172.21.15.99:0/1681226596 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5001c040 con 0x55613fef98c0 2024-05-14T17:05:06.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc417fa700 1 -- 172.21.15.99:0/1681226596 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc500a93a0 con 0x55613fef98c0 2024-05-14T17:05:06.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.290+0000 7efc417fa700 1 -- 172.21.15.99:0/1681226596 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc500a9d50 con 0x55613fef98c0 2024-05-14T17:05:06.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 -- 172.21.15.99:0/1681226596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef98c0 msgr2=0x55613fefb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:06.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef98c0 0x55613fefb6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c5b0 tx=0x7efc500cc040 comp rx=0 tx=0).stop 2024-05-14T17:05:06.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 -- 172.21.15.99:0/1681226596 shutdown_connections 2024-05-14T17:05:06.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613fefbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef5760 0x55613fee6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1681226596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef98c0 0x55613fefb6a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 -- 172.21.15.99:0/1681226596 >> 172.21.15.99:0/1681226596 conn(0x55613fe1df40 msgr2=0x55613fef3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:06.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 -- 172.21.15.99:0/1681226596 shutdown_connections 2024-05-14T17:05:06.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 -- 172.21.15.99:0/1681226596 wait complete. 2024-05-14T17:05:06.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.291+0000 7efc6c845b80 1 -- start start 2024-05-14T17:05:06.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 Processor -- start 2024-05-14T17:05:06.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef5760 0x55613ff0ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613ff0b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613ff7d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1aa050 con 0x55613fef98c0 2024-05-14T17:05:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1ab4c0 con 0x55613fefa1e0 2024-05-14T17:05:06.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1aa720 con 0x55613fef5760 2024-05-14T17:05:06.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc6c845b80 10 client.0 _ll_register_callbacks cb 0x55613fdd9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:06.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613ff0b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc5b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef5760 0x55613ff0ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc5affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613ff0b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34138/0 (socket says 172.21.15.99:34138) 2024-05-14T17:05:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc5affd700 1 -- 172.21.15.99:0/1194094943 learned_addr learned my addr 172.21.15.99:0/1194094943 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.292+0000 7efc5bfff700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613ff7d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5affd700 1 -- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef5760 msgr2=0x55613ff0ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5affd700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef5760 0x55613ff0ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5affd700 1 -- 172.21.15.99:0/1194094943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 msgr2=0x55613ff7d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5affd700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613ff7d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5affd700 1 -- 172.21.15.99:0/1194094943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613ff7da50 con 0x55613fef98c0 2024-05-14T17:05:06.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef5760 0x55613ff0ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc5affd700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613ff0b370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc500a9820 tx=0x7efc500a9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f1414b8428d328 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:06.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc3affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:06.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc3affd700 1 -- 172.21.15.99:0/1194094943 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5001c070 con 0x55613fef98c0 2024-05-14T17:05:06.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc3affd700 1 -- 172.21.15.99:0/1194094943 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efc500b2470 con 0x55613fef98c0 2024-05-14T17:05:06.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc500a93a0 con 0x55613fef98c0 2024-05-14T17:05:06.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc3affd700 1 -- 172.21.15.99:0/1194094943 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc500a9890 con 0x55613fef98c0 2024-05-14T17:05:06.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc6c845b80 10 client.15453 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55613ff7ec40 con 0x55613fef98c0 2024-05-14T17:05:06.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc3affd700 1 -- 172.21.15.99:0/1194094943 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc500a93a0 con 0x55613fef98c0 2024-05-14T17:05:07.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc3affd700 1 -- 172.21.15.99:0/1194094943 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efc500bece0 con 0x55613fef98c0 2024-05-14T17:05:07.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc6c845b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc6c845b80 10 client.15453 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:05:07.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.293+0000 7efc6c845b80 20 client.15453 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:07.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.294+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff7f2d0 0x55613ff816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:07.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.294+0000 7efc6c845b80 4 client.15453 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.2, "await_for": 45.0}] 2024-05-14T17:05:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.294+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.2, "await_for": 45.0}) -- 0x55613fe4a630 con 0x55613ff7f2d0 2024-05-14T17:05:07.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.294+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff7f2d0 0x55613ff816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:07.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.294+0000 7efc3affd700 10 client.15453 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:07.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.294+0000 7efc5b7fe700 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff7f2d0 0x55613ff816f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ecd73fe09f6052 server_cookie=72a6051b989c4a4b in_seq=0 out_seq=0 2024-05-14T17:05:07.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc3affd700 1 -- 172.21.15.99:0/1194094943 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55613fe4a630 con 0x55613ff7f2d0 2024-05-14T17:05:07.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc3affd700 10 client.15453 handle_command_reply: tid=0 2024-05-14T17:05:07.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 client.15453 shutdown 2024-05-14T17:05:07.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 msgr2=0x55613ff0b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:07.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613ff0b370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc500a9820 tx=0x7efc500a9da0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 shutdown_connections 2024-05-14T17:05:07.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55613fefa1e0 0x55613ff7d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55613fef5760 0x55613ff0ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55613ff7f2d0 0x55613ff816f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 --2- 172.21.15.99:0/1194094943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55613fef98c0 0x55613ff0b370 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 >> 172.21.15.99:0/1194094943 conn(0x55613fe1df40 msgr2=0x55613fef3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc3affd700 0 client.15453 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:07.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 shutdown_connections 2024-05-14T17:05:07.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 1 -- 172.21.15.99:0/1194094943 wait complete. 2024-05-14T17:05:07.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.304+0000 7efc6c845b80 20 client.15453 trim_cache size 0 max 16384 2024-05-14T17:05:07.392 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:05:07.405 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2024-05-14T17:05:07.420 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:05:07.444 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 2, 2024-05-14T17:05:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:05:07.545 INFO:teuthology.orchestra.run.smithi099.stdout: "3d220c65": { 2024-05-14T17:05:07.560 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 2, 2024-05-14T17:05:07.577 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:05:07.585 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:05:07.600 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:05:07.614 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:05:07.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:05:07.643 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:05:07.657 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 135.2, 2024-05-14T17:05:07.672 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:05:07.686 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:05:07.697 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:05:07.707 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:05:07.719 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:05:07.730 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:05:07.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:07.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:07.953 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:05:07.962 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:04 smithi193 ceph-mon[35962]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T17:05:07.971 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:06 smithi193 ceph-mon[35962]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T17:05:07.978 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:04 smithi150 ceph-mon[36204]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T17:05:07.989 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:06 smithi150 ceph-mon[36204]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T17:05:08.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efc440b2c40 msgr2=0x7efc440b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:08.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efc440b2c40 0x7efc440b5100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc4c003760 tx=0x7efc4c005ad0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c103f80 msgr2=0x7efc5c19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:08.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c103f80 0x7efc5c19fc20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e9f0 tx=0x7efc5000ed00 comp rx=0 tx=0).stop 2024-05-14T17:05:08.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 shutdown_connections 2024-05-14T17:05:08.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efc440b2c40 0x7efc440b5100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c109630 0x7efc5c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc5c1035c0 0x7efc5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 --2- 172.21.15.99:0/2254742347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc5c103f80 0x7efc5c19fc20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 >> 172.21.15.99:0/2254742347 conn(0x7efc5c07a660 msgr2=0x7efc5c102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 shutdown_connections 2024-05-14T17:05:08.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:00.305+0000 7efc61faf700 1 -- 172.21.15.99:0/2254742347 wait complete. 2024-05-14T17:05:08.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:00 smithi099 ceph-mon[31851]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-14T17:05:08.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:02 smithi099 ceph-mon[31851]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-14T17:05:08.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:04 smithi099 ceph-mon[31851]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T17:05:08.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:06 smithi099 ceph-mon[31851]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T17:05:08.240 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3d220c65', quiesce took 13.0 seconds. Will release after: 2.1999999999999993 2024-05-14T17:05:09.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:08 smithi150 ceph-mon[36204]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T17:05:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:08 smithi099 ceph-mon[31851]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T17:05:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:08 smithi193 ceph-mon[35962]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T17:05:10.450 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:05:10.722 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:10 smithi150 ceph-mon[36204]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-14T17:05:11.285 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:10 smithi150 ceph-mon[36204]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-14T17:05:11.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:10 smithi193 ceph-mon[35962]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-14T17:05:11.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:10 smithi193 ceph-mon[35962]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-14T17:05:11.479 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:10 smithi099 ceph-mon[31851]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-14T17:05:11.500 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:10 smithi099 ceph-mon[31851]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-14T17:05:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 -- 172.21.15.99:0/3299354582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 msgr2=0x7f86a8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 --2- 172.21.15.99:0/3299354582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8102ee0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f869c009930 tx=0x7f869c02a420 comp rx=0 tx=0).stop 2024-05-14T17:05:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 -- 172.21.15.99:0/3299354582 shutdown_connections 2024-05-14T17:05:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 --2- 172.21.15.99:0/3299354582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86a8103420 0x7f86a810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 --2- 172.21.15.99:0/3299354582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8102ee0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 --2- 172.21.15.99:0/3299354582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a8108a70 0x7f86a8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 -- 172.21.15.99:0/3299354582 >> 172.21.15.99:0/3299354582 conn(0x7f86a8079ee0 msgr2=0x7f86a807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 -- 172.21.15.99:0/3299354582 shutdown_connections 2024-05-14T17:05:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.895+0000 7f86acea7640 1 -- 172.21.15.99:0/3299354582 wait complete. 2024-05-14T17:05:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 Processor -- start 2024-05-14T17:05:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 -- start start 2024-05-14T17:05:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a8103420 0x7f86a8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86a8108a70 0x7f86a819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8112d00 con 0x7f86a8102a60 2024-05-14T17:05:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8112b80 con 0x7f86a8108a70 2024-05-14T17:05:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86acea7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8112e80 con 0x7f86a8103420 2024-05-14T17:05:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86a659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86a659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42562/0 (socket says 172.21.15.99:42562) 2024-05-14T17:05:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86a659c640 1 -- 172.21.15.99:0/3556470511 learned_addr learned my addr 172.21.15.99:0/3556470511 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a8103420 0x7f86a8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86a8108a70 0x7f86a819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.896+0000 7f86a659c640 1 -- 172.21.15.99:0/3556470511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a8103420 msgr2=0x7f86a8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86a659c640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a8103420 0x7f86a8199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86a659c640 1 -- 172.21.15.99:0/3556470511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86a8108a70 msgr2=0x7f86a819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86a659c640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86a8108a70 0x7f86a819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86a659c640 1 -- 172.21.15.99:0/3556470511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a819e1a0 con 0x7f86a8102a60 2024-05-14T17:05:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86a659c640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8199200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f869800d950 tx=0x7f869800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86977fe640 1 -- 172.21.15.99:0/3556470511 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698017030 con 0x7f86a8102a60 2024-05-14T17:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86977fe640 1 -- 172.21.15.99:0/3556470511 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8698004020 con 0x7f86a8102a60 2024-05-14T17:05:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a819e490 con 0x7f86a8102a60 2024-05-14T17:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86977fe640 1 -- 172.21.15.99:0/3556470511 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980047a0 con 0x7f86a8102a60 2024-05-14T17:05:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.897+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a81a5d20 con 0x7f86a8102a60 2024-05-14T17:05:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.900+0000 7f86afbca740 1 Processor -- start 2024-05-14T17:05:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86977fe640 1 -- 172.21.15.99:0/3556470511 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8698009a30 con 0x7f86a8102a60 2024-05-14T17:05:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 -- start start 2024-05-14T17:05:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b381bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b3909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f89b2f4c30 con 0x55f89b390ee0 2024-05-14T17:05:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f89b2e47d0 con 0x55f89b382110 2024-05-14T17:05:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86afbca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f89b2e8810 con 0x55f89b3817f0 2024-05-14T17:05:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86a5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86a5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3932d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42568/0 (socket says 172.21.15.99:42568) 2024-05-14T17:05:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86a5d9b640 1 -- 172.21.15.99:0/2070433875 learned_addr learned my addr 172.21.15.99:0/2070433875 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b381bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.901+0000 7f86a659c640 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b3909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a5d9b640 1 -- 172.21.15.99:0/2070433875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 msgr2=0x55f89b381bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b381bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a5d9b640 1 -- 172.21.15.99:0/2070433875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 msgr2=0x55f89b3909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b3909a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a5d9b640 1 -- 172.21.15.99:0/2070433875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89aff9210 con 0x55f89b390ee0 2024-05-14T17:05:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86977fe640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f86900b2a10 0x7f86900b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f86900b2a10 0x7f86900b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3932d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f869c02a940 tx=0x7f869c031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19289f678f5065e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86977fe640 1 -- 172.21.15.99:0/3556470511 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f86980e1fb0 con 0x7f86a8102a60 2024-05-14T17:05:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f8694ff9640 1 -- 172.21.15.99:0/2070433875 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c040070 con 0x55f89b390ee0 2024-05-14T17:05:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f86900b2a10 0x7f86900b4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005fd0 tx=0x7f86a0009500 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f8694ff9640 1 -- 172.21.15.99:0/2070433875 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869c0316a0 con 0x55f89b390ee0 2024-05-14T17:05:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.902+0000 7f8694ff9640 1 -- 172.21.15.99:0/2070433875 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c035040 con 0x55f89b390ee0 2024-05-14T17:05:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 -- 172.21.15.99:0/2070433875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 msgr2=0x55f89b3932d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3932d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f869c02a940 tx=0x7f869c031bc0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 -- 172.21.15.99:0/2070433875 shutdown_connections 2024-05-14T17:05:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3932d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b3909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 --2- 172.21.15.99:0/2070433875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b381bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 -- 172.21.15.99:0/2070433875 >> 172.21.15.99:0/2070433875 conn(0x55f89b37de00 msgr2=0x55f89b37e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 -- 172.21.15.99:0/2070433875 shutdown_connections 2024-05-14T17:05:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 -- 172.21.15.99:0/2070433875 wait complete. 2024-05-14T17:05:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.903+0000 7f86afbca740 1 -- start start 2024-05-14T17:05:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 Processor -- start 2024-05-14T17:05:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b390750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 0x55f89b38da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b390ee0 0x55f89b38df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f89afca790 con 0x55f89b382110 2024-05-14T17:05:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f89b2a0890 con 0x55f89b390ee0 2024-05-14T17:05:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f89b393b50 con 0x55f89b3817f0 2024-05-14T17:05:14.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86afbca740 10 client.0 _ll_register_callbacks cb 0x55f89b2a3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 0x55f89b38da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 0x55f89b38da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42580/0 (socket says 172.21.15.99:42580) 2024-05-14T17:05:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 -- 172.21.15.99:0/1437212502 learned_addr learned my addr 172.21.15.99:0/1437212502 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a659c640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b390750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b390ee0 0x55f89b38df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 -- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 msgr2=0x55f89b390750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b390750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 -- 172.21.15.99:0/1437212502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b390ee0 msgr2=0x55f89b38df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b390ee0 0x55f89b38df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 -- 172.21.15.99:0/1437212502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89b0c5620 con 0x55f89b382110 2024-05-14T17:05:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a659c640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b390750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:14.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b390ee0 0x55f89b38df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:14.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 0x55f89b38da40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f869c031440 tx=0x7f869c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216d55059c1d09ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.904+0000 7f868dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:14.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f868dffb640 1 -- 172.21.15.99:0/1437212502 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c035070 con 0x55f89b382110 2024-05-14T17:05:14.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f868dffb640 1 -- 172.21.15.99:0/1437212502 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f869c005510 con 0x55f89b382110 2024-05-14T17:05:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869c0316a0 con 0x55f89b382110 2024-05-14T17:05:14.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f868dffb640 1 -- 172.21.15.99:0/1437212502 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f869c055050 con 0x55f89b382110 2024-05-14T17:05:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f89b39f4f0 con 0x55f89b382110 2024-05-14T17:05:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f868dffb640 1 -- 172.21.15.99:0/1437212502 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f869c0316a0 con 0x55f89b382110 2024-05-14T17:05:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f868dffb640 1 -- 172.21.15.99:0/1437212502 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f869c0035f0 con 0x55f89b382110 2024-05-14T17:05:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 20 client.15465 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b3a0840 0x55f89b3a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 4 client.15465 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.905+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f89b2f6b80 con 0x55f89b3a0840 2024-05-14T17:05:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.906+0000 7f86a659c640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b3a0840 0x55f89b3a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.906+0000 7f86a659c640 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b3a0840 0x55f89b3a2c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c2de7c162f2476 server_cookie=393d867087952c08 in_seq=0 out_seq=0 2024-05-14T17:05:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.906+0000 7f868dffb640 10 client.15465 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f868dffb640 1 -- 172.21.15.99:0/1437212502 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f89b2f6b80 con 0x55f89b3a0840 2024-05-14T17:05:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f868dffb640 10 client.15465 handle_command_reply: tid=0 2024-05-14T17:05:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 client.15465 shutdown 2024-05-14T17:05:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 msgr2=0x55f89b38da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 0x55f89b38da40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f869c031440 tx=0x7f869c005020 comp rx=0 tx=0).stop 2024-05-14T17:05:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 shutdown_connections 2024-05-14T17:05:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b3a0840 0x55f89b3a2c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b390ee0 0x55f89b38df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b382110 0x55f89b38da40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f868dffb640 0 client.15465 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 --2- 172.21.15.99:0/1437212502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b390750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 >> 172.21.15.99:0/1437212502 conn(0x55f89b37de00 msgr2=0x55f89b383c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 shutdown_connections 2024-05-14T17:05:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 1 -- 172.21.15.99:0/1437212502 wait complete. 2024-05-14T17:05:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.907+0000 7f86afbca740 20 client.15465 trim_cache size 0 max 16384 2024-05-14T17:05:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 Processor -- start 2024-05-14T17:05:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 -- start start 2024-05-14T17:05:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b37d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b390280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81aec90 con 0x55f89b390ee0 2024-05-14T17:05:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81ac560 con 0x55f89b382110 2024-05-14T17:05:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86afbca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81aa3b0 con 0x55f89b3817f0 2024-05-14T17:05:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86a6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86a6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42586/0 (socket says 172.21.15.99:42586) 2024-05-14T17:05:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.938+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2734311839 learned_addr learned my addr 172.21.15.99:0/2734311839 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a659c640 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b37d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b390280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2734311839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 msgr2=0x55f89b37d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b37d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2734311839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 msgr2=0x55f89b390280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b390280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2734311839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89b37dc30 con 0x55f89b390ee0 2024-05-14T17:05:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3a8260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f86a0011910 tx=0x7f86a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37449ba3957a63c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f868ffff640 1 -- 172.21.15.99:0/2734311839 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a002e070 con 0x55f89b390ee0 2024-05-14T17:05:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f868ffff640 1 -- 172.21.15.99:0/2734311839 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a0029410 con 0x55f89b390ee0 2024-05-14T17:05:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.939+0000 7f868ffff640 1 -- 172.21.15.99:0/2734311839 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0020e60 con 0x55f89b390ee0 2024-05-14T17:05:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 -- 172.21.15.99:0/2734311839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 msgr2=0x55f89b3a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3a8260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f86a0011910 tx=0x7f86a0017000 comp rx=0 tx=0).stop 2024-05-14T17:05:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 -- 172.21.15.99:0/2734311839 shutdown_connections 2024-05-14T17:05:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b3a8260 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b382110 0x55f89b390280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 --2- 172.21.15.99:0/2734311839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b3817f0 0x55f89b37d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 -- 172.21.15.99:0/2734311839 >> 172.21.15.99:0/2734311839 conn(0x55f89b37de00 msgr2=0x55f89b388fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 -- 172.21.15.99:0/2734311839 shutdown_connections 2024-05-14T17:05:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.940+0000 7f86afbca740 1 -- 172.21.15.99:0/2734311839 wait complete. 2024-05-14T17:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 -- start start 2024-05-14T17:05:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 Processor -- start 2024-05-14T17:05:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b3817f0 0x55f89b38aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b382110 0x55f89b38af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b38b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81aaa20 con 0x55f89b390ee0 2024-05-14T17:05:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81abe90 con 0x55f89b3817f0 2024-05-14T17:05:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81ab0f0 con 0x55f89b382110 2024-05-14T17:05:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86afbca740 10 client.0 _ll_register_callbacks cb 0x7f869c031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86a6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b38b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86a6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b38b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42602/0 (socket says 172.21.15.99:42602) 2024-05-14T17:05:15.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2291080579 learned_addr learned my addr 172.21.15.99:0/2291080579 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b382110 0x55f89b38af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.941+0000 7f86a659c640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b3817f0 0x55f89b38aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b382110 msgr2=0x55f89b38af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b382110 0x55f89b38af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2291080579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b3817f0 msgr2=0x55f89b38aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b3817f0 0x55f89b38aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a6d9d640 1 -- 172.21.15.99:0/2291080579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89b40a010 con 0x55f89b390ee0 2024-05-14T17:05:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a5d9b640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b382110 0x55f89b38af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86a6d9d640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b38b4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f86a002c040 tx=0x7f86a0003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6236b355e1964433 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f868dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f868dffb640 1 -- 172.21.15.99:0/2291080579 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a002e070 con 0x55f89b390ee0 2024-05-14T17:05:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f868dffb640 1 -- 172.21.15.99:0/2291080579 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f86a0004c50 con 0x55f89b390ee0 2024-05-14T17:05:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a0029410 con 0x55f89b390ee0 2024-05-14T17:05:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f868dffb640 1 -- 172.21.15.99:0/2291080579 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86a0004e50 con 0x55f89b390ee0 2024-05-14T17:05:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86afbca740 10 client.15471 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f89b40b2a0 con 0x55f89b390ee0 2024-05-14T17:05:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.942+0000 7f868dffb640 1 -- 172.21.15.99:0/2291080579 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a0029410 con 0x55f89b390ee0 2024-05-14T17:05:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f868dffb640 1 -- 172.21.15.99:0/2291080579 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f86a0004040 con 0x55f89b390ee0 2024-05-14T17:05:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86afbca740 10 client.15471 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86afbca740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86afbca740 20 client.15471 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86afbca740 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b40b9e0 0x55f89b40de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86afbca740 4 client.15471 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f89b2f0360 con 0x55f89b40b9e0 2024-05-14T17:05:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86a659c640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b40b9e0 0x55f89b40de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f868dffb640 10 client.15471 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.943+0000 7f86a659c640 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b40b9e0 0x55f89b40de00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e0723a995757a3 server_cookie=71de004984536eca in_seq=0 out_seq=0 2024-05-14T17:05:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f868dffb640 1 -- 172.21.15.99:0/2291080579 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f89b2f0360 con 0x55f89b40b9e0 2024-05-14T17:05:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f868dffb640 10 client.15471 handle_command_reply: tid=0 2024-05-14T17:05:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f86afbca740 1 client.15471 shutdown 2024-05-14T17:05:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 msgr2=0x55f89b38b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f86afbca740 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b38b4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f86a002c040 tx=0x7f86a0003dc0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 shutdown_connections 2024-05-14T17:05:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.950+0000 7f86afbca740 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f89b40b9e0 0x55f89b40de00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f89b390ee0 0x55f89b38b4c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f89b382110 0x55f89b38af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 1 --2- 172.21.15.99:0/2291080579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f89b3817f0 0x55f89b38aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 >> 172.21.15.99:0/2291080579 conn(0x55f89b37de00 msgr2=0x55f89b37d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f868dffb640 0 client.15471 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 shutdown_connections 2024-05-14T17:05:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 1 -- 172.21.15.99:0/2291080579 wait complete. 2024-05-14T17:05:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86afbca740 20 client.15471 trim_cache size 0 max 16384 2024-05-14T17:05:16.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:05:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:05:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:05:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f86900b2a10 msgr2=0x7f86900b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86acea7640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f86900b2a10 0x7f86900b4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005fd0 tx=0x7f86a0009500 comp rx=0 tx=0).stop 2024-05-14T17:05:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 msgr2=0x7f86a8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.951+0000 7f86acea7640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8199200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f869800d950 tx=0x7f869800de10 comp rx=0 tx=0).stop 2024-05-14T17:05:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 shutdown_connections 2024-05-14T17:05:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f86900b2a10 0x7f86900b4ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86a8108a70 0x7f86a819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a8103420 0x7f86a8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 --2- 172.21.15.99:0/3556470511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a8102a60 0x7f86a8199200 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 >> 172.21.15.99:0/3556470511 conn(0x7f86a8079ee0 msgr2=0x7f86a8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 shutdown_connections 2024-05-14T17:05:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:12.952+0000 7f86acea7640 1 -- 172.21.15.99:0/3556470511 wait complete. 2024-05-14T17:05:16.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:12 smithi099 ceph-mon[31851]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-14T17:05:16.672 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:12 smithi193 ceph-mon[35962]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-14T17:05:16.688 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:13 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/166864640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:16.699 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:14 smithi193 ceph-mon[35962]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-14T17:05:16.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:12 smithi150 ceph-mon[36204]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-14T17:05:16.732 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:13 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/166864640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:16.744 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:14 smithi150 ceph-mon[36204]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-14T17:05:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 -- 172.21.15.99:0/2886406106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24108520 msgr2=0x7f1d24108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:16.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/2886406106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24108520 0x7f1d24108900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009990 tx=0x7f1d14009ca0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 -- 172.21.15.99:0/2886406106 shutdown_connections 2024-05-14T17:05:16.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/2886406106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24102e00 0x7f1d2410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/2886406106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24108520 0x7f1d24108900 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/2886406106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d24102440 0x7f1d241028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:16.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 -- 172.21.15.99:0/2886406106 >> 172.21.15.99:0/2886406106 conn(0x7f1d240793f0 msgr2=0x7f1d24079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:16.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 -- 172.21.15.99:0/2886406106 shutdown_connections 2024-05-14T17:05:16.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.457+0000 7f1d29e9f700 1 -- 172.21.15.99:0/2886406106 wait complete. 2024-05-14T17:05:16.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 Processor -- start 2024-05-14T17:05:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 -- start start 2024-05-14T17:05:16.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d24102440 0x7f1d24109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:16.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24102e00 0x7f1d241098d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24108520 0x7f1d24109e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d241125f0 con 0x7f1d24102e00 2024-05-14T17:05:17.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24112470 con 0x7f1d24108520 2024-05-14T17:05:17.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d29e9f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24112770 con 0x7f1d24102440 2024-05-14T17:05:17.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d22ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24102e00 0x7f1d241098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d23fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24108520 0x7f1d24109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d22ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24102e00 0x7f1d241098d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42620/0 (socket says 172.21.15.99:42620) 2024-05-14T17:05:17.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.458+0000 7f1d22ffd700 1 -- 172.21.15.99:0/166864640 learned_addr learned my addr 172.21.15.99:0/166864640 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:17.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d237fe700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d24102440 0x7f1d24109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d23fff700 1 -- 172.21.15.99:0/166864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d24102440 msgr2=0x7f1d24109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d23fff700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d24102440 0x7f1d24109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d23fff700 1 -- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24102e00 msgr2=0x7f1d241098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d23fff700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24102e00 0x7f1d241098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d23fff700 1 -- 172.21.15.99:0/166864640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d241a8190 con 0x7f1d24108520 2024-05-14T17:05:17.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d23fff700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24108520 0x7f1d24109e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00efc0 tx=0x7f1d1c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0092e0 con 0x7f1d24108520 2024-05-14T17:05:17.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c004bb0 con 0x7f1d24108520 2024-05-14T17:05:17.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d241a8420 con 0x7f1d24108520 2024-05-14T17:05:17.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c004080 con 0x7f1d24108520 2024-05-14T17:05:17.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.459+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a8910 con 0x7f1d24108520 2024-05-14T17:05:17.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.460+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2406a230 con 0x7f1d24108520 2024-05-14T17:05:17.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.462+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1d1c01b030 con 0x7f1d24108520 2024-05-14T17:05:17.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.462+0000 7f1d20ff9700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1d0c0b29f0 0x7f1d0c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.462+0000 7f1d237fe700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1d0c0b29f0 0x7f1d0c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.462+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f1d1c0e2a90 con 0x7f1d24108520 2024-05-14T17:05:17.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.463+0000 7f1d237fe700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1d0c0b29f0 0x7f1d0c0b4eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d14006010 tx=0x7f1d1401a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.466+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d1c022020 con 0x7f1d24108520 2024-05-14T17:05:17.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.739+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d241a8c40 con 0x7f1d24108520 2024-05-14T17:05:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.740+0000 7f1d20ff9700 1 -- 172.21.15.99:0/166864640 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7f1d1c010040 con 0x7f1d24108520 2024-05-14T17:05:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:05:17.458 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:05:17.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1d0c0b29f0 msgr2=0x7f1d0c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1d0c0b29f0 0x7f1d0c0b4eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d14006010 tx=0x7f1d1401a040 comp rx=0 tx=0).stop 2024-05-14T17:05:17.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24108520 msgr2=0x7f1d24109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24108520 0x7f1d24109e10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00efc0 tx=0x7f1d1c00c3a0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 shutdown_connections 2024-05-14T17:05:17.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1d0c0b29f0 0x7f1d0c0b4eb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d24108520 0x7f1d24109e10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d24102440 0x7f1d24109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 --2- 172.21.15.99:0/166864640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d24102e00 0x7f1d241098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 >> 172.21.15.99:0/166864640 conn(0x7f1d240793f0 msgr2=0x7f1d240fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:17.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 shutdown_connections 2024-05-14T17:05:17.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:13.743+0000 7f1d29e9f700 1 -- 172.21.15.99:0/166864640 wait complete. 2024-05-14T17:05:17.638 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:05:17.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:13 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/166864640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:17.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 3d220c65 --release --await' 2024-05-14T17:05:17.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 3d220c65 --release --await 2024-05-14T17:05:17.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:14 smithi099 ceph-mon[31851]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-14T17:05:17.699 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:16 smithi099 ceph-mon[31851]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-14T17:05:17.711 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:16 smithi193 ceph-mon[35962]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-14T17:05:17.718 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:16 smithi150 ceph-mon[36204]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-14T17:05:17.992 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:19.165 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:18 smithi099 ceph-mon[31851]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-14T17:05:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:18 smithi150 ceph-mon[36204]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-14T17:05:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:18 smithi193 ceph-mon[35962]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-14T17:05:19.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 -- 172.21.15.99:0/4170793955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 msgr2=0x7f8f1810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/4170793955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f1810f2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b0a0 tx=0x7f8f1000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 -- 172.21.15.99:0/4170793955 shutdown_connections 2024-05-14T17:05:19.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/4170793955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f1810f2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/4170793955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f18108520 0x7f8f18108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/4170793955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f18102440 0x7f8f181028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 -- 172.21.15.99:0/4170793955 >> 172.21.15.99:0/4170793955 conn(0x7f8f180793f0 msgr2=0x7f8f18079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:19.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 -- 172.21.15.99:0/4170793955 shutdown_connections 2024-05-14T17:05:19.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.741+0000 7f8f1fd59700 1 -- 172.21.15.99:0/4170793955 wait complete. 2024-05-14T17:05:19.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 Processor -- start 2024-05-14T17:05:19.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 -- start start 2024-05-14T17:05:20.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f18102440 0x7f8f181146f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f18112db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f18108520 0x7f8f181132f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181125f0 con 0x7f8f18102e00 2024-05-14T17:05:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18112470 con 0x7f8f18108520 2024-05-14T17:05:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1fd59700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18112770 con 0x7f8f18102440 2024-05-14T17:05:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1d2f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f18112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1d2f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f18112db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45578/0 (socket says 172.21.15.99:45578) 2024-05-14T17:05:20.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2058550616 learned_addr learned my addr 172.21.15.99:0/2058550616 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:20.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.742+0000 7f8f1e2f6700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f18108520 0x7f8f181132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f18102440 0x7f8f181146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2058550616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f18102440 msgr2=0x7f8f181146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f18102440 0x7f8f181146f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2058550616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f18108520 msgr2=0x7f8f181132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f18108520 0x7f8f181132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2058550616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f181139f0 con 0x7f8f18102e00 2024-05-14T17:05:20.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f18112db0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800e9c0 tx=0x7f8f0800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:20.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f0effd700 1 -- 172.21.15.99:0/2058550616 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800cbd0 con 0x7f8f18102e00 2024-05-14T17:05:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f0effd700 1 -- 172.21.15.99:0/2058550616 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f0800ee60 con 0x7f8f18102e00 2024-05-14T17:05:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.743+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181ac920 con 0x7f8f18102e00 2024-05-14T17:05:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.744+0000 7f8f0effd700 1 -- 172.21.15.99:0/2058550616 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08005580 con 0x7f8f18102e00 2024-05-14T17:05:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.744+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181accc0 con 0x7f8f18102e00 2024-05-14T17:05:20.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.746+0000 7f8f0effd700 1 -- 172.21.15.99:0/2058550616 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8f08004750 con 0x7f8f18102e00 2024-05-14T17:05:20.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.747+0000 7f8f0effd700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8f040b29f0 0x7f8f040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.748+0000 7f8f0effd700 1 -- 172.21.15.99:0/2058550616 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f8f080e2940 con 0x7f8f18102e00 2024-05-14T17:05:20.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.748+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8f040b29f0 0x7f8f040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.748+0000 7f8f2a5efb80 1 Processor -- start 2024-05-14T17:05:20.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.748+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8f040b29f0 0x7f8f040b4eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007b60 tx=0x7f8f140073d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:20.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.748+0000 7f8f2a5efb80 1 -- start start 2024-05-14T17:05:20.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f1d2f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f1d2f4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4ef4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45586/0 (socket says 172.21.15.99:45586) 2024-05-14T17:05:20.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f1daf5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f1daf5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:52670/0 (socket says 172.21.15.99:52670) 2024-05-14T17:05:20.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/1596832124 learned_addr learned my addr 172.21.15.99:0/1596832124 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55685f44dd90 con 0x55685f4f56c0 2024-05-14T17:05:20.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55685f43d930 con 0x55685f4f4da0 2024-05-14T17:05:20.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.749+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55685f441970 con 0x55685f4efad0 2024-05-14T17:05:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/1596832124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 msgr2=0x55685f4eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:20.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/1596832124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 msgr2=0x55685f4f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/1596832124 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685f38d9d0 con 0x55685f4f56c0 2024-05-14T17:05:20.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4ef4b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8f080a9950 tx=0x7f8f08005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd874c03054a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:20.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f03fff700 1 -- 172.21.15.99:0/1596832124 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08091040 con 0x55685f4f56c0 2024-05-14T17:05:20.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f03fff700 1 -- 172.21.15.99:0/1596832124 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f080b2400 con 0x55685f4f56c0 2024-05-14T17:05:20.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.750+0000 7f8f03fff700 1 -- 172.21.15.99:0/1596832124 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f080aa110 con 0x55685f4f56c0 2024-05-14T17:05:20.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 msgr2=0x55685f4ef4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4ef4b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8f080a9950 tx=0x7f8f08005740 comp rx=0 tx=0).stop 2024-05-14T17:05:20.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 shutdown_connections 2024-05-14T17:05:20.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1596832124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4ef4b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 >> 172.21.15.99:0/1596832124 conn(0x55685f41a350 msgr2=0x55685f4e9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.751+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 shutdown_connections 2024-05-14T17:05:20.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.752+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1596832124 wait complete. 2024-05-14T17:05:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.752+0000 7f8f2a5efb80 1 -- start start 2024-05-14T17:05:20.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.752+0000 7f8f2a5efb80 1 Processor -- start 2024-05-14T17:05:20.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4efad0 0x55685f4fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f4da0 0x55685f4f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55685f3440a0 con 0x55685f4f56c0 2024-05-14T17:05:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55685f3df8c0 con 0x55685f4efad0 2024-05-14T17:05:20.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55685f4fc100 con 0x55685f4f4da0 2024-05-14T17:05:20.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f2a5efb80 10 client.0 _ll_register_callbacks cb 0x55685f3d4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:20.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1e2f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1e2f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4f8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45588/0 (socket says 172.21.15.99:45588) 2024-05-14T17:05:20.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1e2f6700 1 -- 172.21.15.99:0/2699734829 learned_addr learned my addr 172.21.15.99:0/2699734829 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:20.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f4da0 0x55685f4f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4efad0 0x55685f4fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2699734829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4efad0 msgr2=0x55685f4fb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:21.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4efad0 0x55685f4fb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2699734829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 msgr2=0x55685f4f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:21.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1d2f4700 1 -- 172.21.15.99:0/2699734829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685f3dc790 con 0x55685f4f4da0 2024-05-14T17:05:21.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.753+0000 7f8f1e2f6700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:21.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f4da0 0x55685f4f87b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f080a9610 tx=0x7f8f080a5460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4fdc6294dc59d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:21.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f017fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f017fa700 1 -- 172.21.15.99:0/2699734829 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08091040 con 0x55685f4f4da0 2024-05-14T17:05:21.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f017fa700 1 -- 172.21.15.99:0/2699734829 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f8f080b2830 con 0x55685f4f4da0 2024-05-14T17:05:21.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f080b2400 con 0x55685f4f4da0 2024-05-14T17:05:21.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f017fa700 1 -- 172.21.15.99:0/2699734829 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f080aa350 con 0x55685f4f4da0 2024-05-14T17:05:21.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f2a5efb80 10 client.25084 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:21.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.754+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55685f506850 con 0x55685f4f4da0 2024-05-14T17:05:21.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f017fa700 1 -- 172.21.15.99:0/2699734829 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f080b2400 con 0x55685f4f4da0 2024-05-14T17:05:21.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f017fa700 1 -- 172.21.15.99:0/2699734829 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f080a2af0 con 0x55685f4f4da0 2024-05-14T17:05:21.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f2a5efb80 10 client.25084 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:21.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f2a5efb80 10 client.25084 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:05:21.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f2a5efb80 20 client.25084 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:21.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f508d20 0x55685f50b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f2a5efb80 4 client.25084 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55685f4bb460 con 0x55685f508d20 2024-05-14T17:05:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.755+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f508d20 0x55685f50b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.756+0000 7f8f017fa700 10 client.25084 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.756+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f508d20 0x55685f50b110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0735c57e9b54b2 server_cookie=39bf35cc09cdaf80 in_seq=0 out_seq=0 2024-05-14T17:05:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.756+0000 7f8f017fa700 1 -- 172.21.15.99:0/2699734829 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55685f4bb460 con 0x55685f508d20 2024-05-14T17:05:21.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.756+0000 7f8f017fa700 10 client.25084 handle_command_reply: tid=0 2024-05-14T17:05:21.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.756+0000 7f8f2a5efb80 1 client.25084 shutdown 2024-05-14T17:05:21.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f4da0 msgr2=0x55685f4f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:21.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f4da0 0x55685f4f87b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f080a9610 tx=0x7f8f080a5460 comp rx=0 tx=0).stop 2024-05-14T17:05:21.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 shutdown_connections 2024-05-14T17:05:21.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f508d20 0x55685f50b110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f4f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4efad0 0x55685f4fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/2699734829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f4da0 0x55685f4f87b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 >> 172.21.15.99:0/2699734829 conn(0x55685f41a350 msgr2=0x55685f4e47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:21.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f017fa700 0 client.25084 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:21.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 shutdown_connections 2024-05-14T17:05:21.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/2699734829 wait complete. 2024-05-14T17:05:21.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.757+0000 7f8f2a5efb80 20 client.25084 trim_cache size 0 max 16384 2024-05-14T17:05:21.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 Processor -- start 2024-05-14T17:05:21.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 -- start start 2024-05-14T17:05:21.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f504aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181ae410 con 0x55685f4f56c0 2024-05-14T17:05:21.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181a55f0 con 0x55685f4f4da0 2024-05-14T17:05:21.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181a3460 con 0x55685f4efad0 2024-05-14T17:05:21.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f1e2f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f504aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f1e2f6700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f504aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45598/0 (socket says 172.21.15.99:45598) 2024-05-14T17:05:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.794+0000 7f8f1e2f6700 1 -- 172.21.15.99:0/87793367 learned_addr learned my addr 172.21.15.99:0/87793367 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:21.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1e2f6700 1 -- 172.21.15.99:0/87793367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 msgr2=0x55685f4ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:21.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1e2f6700 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1e2f6700 1 -- 172.21.15.99:0/87793367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 msgr2=0x55685f4e48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:21.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1e2f6700 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4e48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1e2f6700 1 -- 172.21.15.99:0/87793367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685f4f6d20 con 0x55685f4f56c0 2024-05-14T17:05:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:21.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f1e2f6700 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f504aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009610 tx=0x7f8f1000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a15b46145957c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:21.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f037fe700 1 -- 172.21.15.99:0/87793367 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10003c00 con 0x55685f4f56c0 2024-05-14T17:05:21.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f037fe700 1 -- 172.21.15.99:0/87793367 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f10003da0 con 0x55685f4f56c0 2024-05-14T17:05:21.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.795+0000 7f8f037fe700 1 -- 172.21.15.99:0/87793367 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10005400 con 0x55685f4f56c0 2024-05-14T17:05:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/87793367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 msgr2=0x55685f504aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:21.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f504aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009610 tx=0x7f8f1000bcf0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/87793367 shutdown_connections 2024-05-14T17:05:21.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4f56c0 0x55685f504aa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4efad0 0x55685f4ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/87793367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f4e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/87793367 >> 172.21.15.99:0/87793367 conn(0x55685f41a350 msgr2=0x55685f504390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:21.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/87793367 shutdown_connections 2024-05-14T17:05:21.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.796+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/87793367 wait complete. 2024-05-14T17:05:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 -- start start 2024-05-14T17:05:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 Processor -- start 2024-05-14T17:05:21.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 0x55685f50dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f50e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f56c0 0x55685f50c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181a3ab0 con 0x55685f4efad0 2024-05-14T17:05:22.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181a4f20 con 0x55685f4f4da0 2024-05-14T17:05:22.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f181a4180 con 0x55685f4f56c0 2024-05-14T17:05:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f2a5efb80 10 client.0 _ll_register_callbacks cb 0x55685f3d4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.797+0000 7f8f1daf5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 0x55685f50dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 0x55685f50dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45606/0 (socket says 172.21.15.99:45606) 2024-05-14T17:05:22.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 -- 172.21.15.99:0/1723502906 learned_addr learned my addr 172.21.15.99:0/1723502906 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:22.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f50e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1e2f6700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f56c0 0x55685f50c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 -- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f56c0 msgr2=0x55685f50c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f56c0 0x55685f50c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 -- 172.21.15.99:0/1723502906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 msgr2=0x55685f50e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:22.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f50e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 -- 172.21.15.99:0/1723502906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685f50c850 con 0x55685f4efad0 2024-05-14T17:05:22.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f50e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:22.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f1daf5700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 0x55685f50dc00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007940 tx=0x7f8f140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a51e0dc21ce35c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:22.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f017fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:22.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f017fa700 1 -- 172.21.15.99:0/1723502906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14030060 con 0x55685f4efad0 2024-05-14T17:05:22.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f017fa700 1 -- 172.21.15.99:0/1723502906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f8f14004460 con 0x55685f4efad0 2024-05-14T17:05:22.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.798+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f10003da0 con 0x55685f4efad0 2024-05-14T17:05:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f017fa700 1 -- 172.21.15.99:0/1723502906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f140118a0 con 0x55685f4efad0 2024-05-14T17:05:22.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 10 client.15495 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:22.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55685f573d00 con 0x55685f4efad0 2024-05-14T17:05:22.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f017fa700 1 -- 172.21.15.99:0/1723502906 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f10003da0 con 0x55685f4efad0 2024-05-14T17:05:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f017fa700 1 -- 172.21.15.99:0/1723502906 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f14003680 con 0x55685f4efad0 2024-05-14T17:05:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:22.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 10 client.15495 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:05:22.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 20 client.15495 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:22.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f5743f0 0x55685f576810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 4 client.15495 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3d220c65", "release": true, "await": true}] 2024-05-14T17:05:22.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.799+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3d220c65", "release": true, "await": true}) -- 0x55685f44fce0 con 0x55685f5743f0 2024-05-14T17:05:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.800+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f5743f0 0x55685f576810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.800+0000 7f8f017fa700 10 client.15495 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:22.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.800+0000 7f8f1d2f4700 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f5743f0 0x55685f576810 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfee672479cfc08 server_cookie=2c679c775b0511e4 in_seq=0 out_seq=0 2024-05-14T17:05:22.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f017fa700 1 -- 172.21.15.99:0/1723502906 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55685f44fce0 con 0x55685f5743f0 2024-05-14T17:05:22.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f017fa700 10 client.15495 handle_command_reply: tid=0 2024-05-14T17:05:22.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 client.15495 shutdown 2024-05-14T17:05:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 msgr2=0x55685f50dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 0x55685f50dc00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007940 tx=0x7f8f140047c0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 shutdown_connections 2024-05-14T17:05:22.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55685f4f56c0 0x55685f50c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55685f5743f0 0x55685f576810 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55685f4efad0 0x55685f50dc00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 --2- 172.21.15.99:0/1723502906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55685f4f4da0 0x55685f50e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 >> 172.21.15.99:0/1723502906 conn(0x55685f41a350 msgr2=0x55685f504c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f017fa700 0 client.15495 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:22.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 shutdown_connections 2024-05-14T17:05:22.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 1 -- 172.21.15.99:0/1723502906 wait complete. 2024-05-14T17:05:22.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.802+0000 7f8f2a5efb80 20 client.15495 trim_cache size 0 max 16384 2024-05-14T17:05:22.680 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:05:22.687 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2024-05-14T17:05:22.693 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:05:22.699 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 4, 2024-05-14T17:05:22.708 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:05:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "3d220c65": { 2024-05-14T17:05:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 4, 2024-05-14T17:05:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:05:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:05:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:05:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:05:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:05:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:05:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 135.2, 2024-05-14T17:05:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:05:22.851 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:05:22.889 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:05:22.895 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:05:22.903 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:05:22.910 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:05:22.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:22.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:22.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:22.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:22.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:22.960 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:05:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8f040b29f0 msgr2=0x7f8f040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:22.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8f040b29f0 0x7f8f040b4eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f14007b60 tx=0x7f8f140073d0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 msgr2=0x7f8f18112db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f18112db0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800e9c0 tx=0x7f8f0800ecd0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 shutdown_connections 2024-05-14T17:05:23.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8f040b29f0 0x7f8f040b4eb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f18108520 0x7f8f181132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f18102440 0x7f8f181146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 --2- 172.21.15.99:0/2058550616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f18102e00 0x7f8f18112db0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:23.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.803+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 >> 172.21.15.99:0/2058550616 conn(0x7f8f180793f0 msgr2=0x7f8f180ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:23.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.804+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 shutdown_connections 2024-05-14T17:05:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:19.804+0000 7f8f1fd59700 1 -- 172.21.15.99:0/2058550616 wait complete. 2024-05-14T17:05:23.155 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:20 smithi099 ceph-mon[31851]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-14T17:05:23.163 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3d220c65', total seconds elapsed: 27.9 2024-05-14T17:05:23.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:20 smithi193 ceph-mon[35962]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-14T17:05:23.179 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:20 smithi150 ceph-mon[36204]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-14T17:05:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:22 smithi150 ceph-mon[36204]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T17:05:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:22 smithi099 ceph-mon[31851]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T17:05:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:22 smithi193 ceph-mon[35962]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T17:05:25.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:24 smithi150 ceph-mon[36204]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:05:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:24 smithi099 ceph-mon[31851]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:05:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:24 smithi193 ceph-mon[35962]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T17:05:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/1379785183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102340 msgr2=0x7ff7f810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/1379785183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102340 0x7ff7f810e800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009cd0 tx=0x7ff7f002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/1379785183 shutdown_connections 2024-05-14T17:05:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/1379785183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102340 0x7ff7f810e800 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/1379785183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f8101980 0x7ff7f8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/1379785183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8107940 0x7ff7f8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.046+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/1379785183 >> 172.21.15.99:0/1379785183 conn(0x7ff7f8078ad0 msgr2=0x7ff7f8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/1379785183 shutdown_connections 2024-05-14T17:05:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/1379785183 wait complete. 2024-05-14T17:05:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 Processor -- start 2024-05-14T17:05:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 -- start start 2024-05-14T17:05:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8101980 0x7ff7f8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f8102340 0x7ff7f819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.047+0000 7ff7ff3a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 0x7ff7f81a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ff3a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111c50 con 0x7ff7f8107940 2024-05-14T17:05:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ff3a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111ad0 con 0x7ff7f8101980 2024-05-14T17:05:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ff3a4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111dd0 con 0x7ff7f8102340 2024-05-14T17:05:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 0x7ff7f81a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd140640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8101980 0x7ff7f8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 0x7ff7f81a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50094/0 (socket says 172.21.15.99:50094) 2024-05-14T17:05:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 -- 172.21.15.99:0/535662401 learned_addr learned my addr 172.21.15.99:0/535662401 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f8102340 0x7ff7f819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 -- 172.21.15.99:0/535662401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f8102340 msgr2=0x7ff7f819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f8102340 0x7ff7f819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 -- 172.21.15.99:0/535662401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8101980 msgr2=0x7ff7f8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8101980 0x7ff7f8199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 -- 172.21.15.99:0/535662401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81a5550 con 0x7ff7f8107940 2024-05-14T17:05:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd140640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8101980 0x7ff7f8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7fd941640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 0x7ff7f81a5010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005e30 tx=0x7ff7f0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ee7fc640 1 -- 172.21.15.99:0/535662401 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0004240 con 0x7ff7f8107940 2024-05-14T17:05:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ee7fc640 1 -- 172.21.15.99:0/535662401 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f0031ce0 con 0x7ff7f8107940 2024-05-14T17:05:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a5720 con 0x7ff7f8107940 2024-05-14T17:05:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.048+0000 7ff7ee7fc640 1 -- 172.21.15.99:0/535662401 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0046480 con 0x7ff7f8107940 2024-05-14T17:05:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.049+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a5c10 con 0x7ff7f8107940 2024-05-14T17:05:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.051+0000 7ff802197740 1 Processor -- start 2024-05-14T17:05:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff7ee7fc640 1 -- 172.21.15.99:0/535662401 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff7f00306a0 con 0x7ff7f8107940 2024-05-14T17:05:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff802197740 1 -- start start 2024-05-14T17:05:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff802197740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b38f0 0x55c9e51b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff802197740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff802197740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff802197740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5126c30 con 0x55c9e51bead0 2024-05-14T17:05:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.052+0000 7ff802197740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c9e51167d0 con 0x55c9e51b4210 2024-05-14T17:05:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff802197740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9e511a810 con 0x55c9e51b38f0 2024-05-14T17:05:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50108/0 (socket says 172.21.15.99:50108) 2024-05-14T17:05:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 -- 172.21.15.99:0/2044522345 learned_addr learned my addr 172.21.15.99:0/2044522345 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b38f0 0x55c9e51b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fd140640 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 -- 172.21.15.99:0/2044522345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b38f0 msgr2=0x55c9e51b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b38f0 0x55c9e51b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 -- 172.21.15.99:0/2044522345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 msgr2=0x55c9e51be590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51be590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 -- 172.21.15.99:0/2044522345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e4e2b210 con 0x55c9e51bead0 2024-05-14T17:05:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fd140640 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7ee7fc640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff7e40b2a90 0x7ff7e40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c0ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c6d0 tx=0x7ff7e800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3630b0ca948dc3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.053+0000 7ff7fd140640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff7e40b2a90 0x7ff7e40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff7ee7fc640 1 -- 172.21.15.99:0/535662401 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7ff7f002e030 con 0x7ff7f8107940 2024-05-14T17:05:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff7e37fe640 1 -- 172.21.15.99:0/2044522345 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8022070 con 0x55c9e51bead0 2024-05-14T17:05:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff7e37fe640 1 -- 172.21.15.99:0/2044522345 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e800fc10 con 0x55c9e51bead0 2024-05-14T17:05:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff7e37fe640 1 -- 172.21.15.99:0/2044522345 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e801da10 con 0x55c9e51bead0 2024-05-14T17:05:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff802197740 1 -- 172.21.15.99:0/2044522345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 msgr2=0x55c9e51c0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff7fd140640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff7e40b2a90 0x7ff7e40b4f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55c9e51b4e90 tx=0x7ff7f400b390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.054+0000 7ff802197740 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c0ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c6d0 tx=0x7ff7e800c4d0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 -- 172.21.15.99:0/2044522345 shutdown_connections 2024-05-14T17:05:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c0ec0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 --2- 172.21.15.99:0/2044522345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b38f0 0x55c9e51b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 -- 172.21.15.99:0/2044522345 >> 172.21.15.99:0/2044522345 conn(0x55c9e51afe00 msgr2=0x55c9e51b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 -- 172.21.15.99:0/2044522345 shutdown_connections 2024-05-14T17:05:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 -- 172.21.15.99:0/2044522345 wait complete. 2024-05-14T17:05:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 -- start start 2024-05-14T17:05:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.055+0000 7ff802197740 1 Processor -- start 2024-05-14T17:05:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 0x55c9e51be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 0x55c9e51be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 0x55c9e51be4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50120/0 (socket says 172.21.15.99:50120) 2024-05-14T17:05:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 -- 172.21.15.99:0/3018934001 learned_addr learned my addr 172.21.15.99:0/3018934001 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51bead0 0x55c9e51bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c9e4dfc790 con 0x55c9e51b38f0 2024-05-14T17:05:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c9e50d2890 con 0x55c9e51b4210 2024-05-14T17:05:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9e51c17a0 con 0x55c9e51bead0 2024-05-14T17:05:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff802197740 10 client.0 _ll_register_callbacks cb 0x55c9e50d5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd941640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51bead0 0x55c9e51bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 -- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51bead0 msgr2=0x55c9e51bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51bead0 0x55c9e51bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 -- 172.21.15.99:0/3018934001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 msgr2=0x55c9e51bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e4ef7620 con 0x55c9e51b38f0 2024-05-14T17:05:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd941640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51bead0 0x55c9e51bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7fd140640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 0x55c9e51be4d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40108f0 tx=0x7ff7f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fefed8841e94e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.056+0000 7ff7e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/3018934001 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f402b060 con 0x55c9e51b38f0 2024-05-14T17:05:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/3018934001 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7ff7f401c840 con 0x55c9e51b38f0 2024-05-14T17:05:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e800fc10 con 0x55c9e51b38f0 2024-05-14T17:05:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/3018934001 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7f401ca40 con 0x55c9e51b38f0 2024-05-14T17:05:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 10 client.15507 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9e51cd280 con 0x55c9e51b38f0 2024-05-14T17:05:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/3018934001 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e800fc10 con 0x55c9e51b38f0 2024-05-14T17:05:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/3018934001 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff7f4025030 con 0x55c9e51b38f0 2024-05-14T17:05:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 20 client.15507 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e51ce800 0x55c9e51d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 4 client.15507 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:27.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.057+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9e5128b80 con 0x55c9e51ce800 2024-05-14T17:05:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.058+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e51ce800 0x55c9e51d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.058+0000 7ff7e0ff9640 10 client.15507 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.058+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e51ce800 0x55c9e51d0c50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33b3e7420193529 server_cookie=aca585921b35107c in_seq=0 out_seq=0 2024-05-14T17:05:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/3018934001 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9e5128b80 con 0x55c9e51ce800 2024-05-14T17:05:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff7e0ff9640 10 client.15507 handle_command_reply: tid=0 2024-05-14T17:05:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 client.15507 shutdown 2024-05-14T17:05:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 msgr2=0x55c9e51be4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 0x55c9e51be4d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40108f0 tx=0x7ff7f40046c0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 shutdown_connections 2024-05-14T17:05:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e51ce800 0x55c9e51d0c50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51bead0 0x55c9e51bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51b4210 0x55c9e51bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 --2- 172.21.15.99:0/3018934001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51b38f0 0x55c9e51be4d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 >> 172.21.15.99:0/3018934001 conn(0x55c9e51afe00 msgr2=0x55c9e51af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff7e0ff9640 0 client.15507 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 shutdown_connections 2024-05-14T17:05:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 1 -- 172.21.15.99:0/3018934001 wait complete. 2024-05-14T17:05:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.059+0000 7ff802197740 20 client.15507 trim_cache size 0 max 16384 2024-05-14T17:05:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 Processor -- start 2024-05-14T17:05:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 -- start start 2024-05-14T17:05:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51bead0 0x55c9e51d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 0x55c9e51b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81aeb60 con 0x55c9e51ab3f0 2024-05-14T17:05:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81ac430 con 0x55c9e51bead0 2024-05-14T17:05:27.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.088+0000 7ff802197740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81aa280 con 0x55c9e51b4210 2024-05-14T17:05:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 0x55c9e51b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 0x55c9e51b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50136/0 (socket says 172.21.15.99:50136) 2024-05-14T17:05:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 -- 172.21.15.99:0/2672177903 learned_addr learned my addr 172.21.15.99:0/2672177903 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd140640 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51bead0 0x55c9e51d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 -- 172.21.15.99:0/2672177903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 msgr2=0x55c9e51d6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51d6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 -- 172.21.15.99:0/2672177903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51bead0 msgr2=0x55c9e51d6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51bead0 0x55c9e51d6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 -- 172.21.15.99:0/2672177903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e51c1230 con 0x55c9e51ab3f0 2024-05-14T17:05:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7fd941640 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 0x55c9e51b8a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0100be0 tx=0x7ff7f00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c8ae9ff161b7ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7e2ffd640 1 -- 172.21.15.99:0/2672177903 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00ca220 con 0x55c9e51ab3f0 2024-05-14T17:05:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7e2ffd640 1 -- 172.21.15.99:0/2672177903 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f0036030 con 0x55c9e51ab3f0 2024-05-14T17:05:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.089+0000 7ff7e2ffd640 1 -- 172.21.15.99:0/2672177903 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00b20e0 con 0x55c9e51ab3f0 2024-05-14T17:05:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 -- 172.21.15.99:0/2672177903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 msgr2=0x55c9e51b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 0x55c9e51b8a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0100be0 tx=0x7ff7f00095c0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 -- 172.21.15.99:0/2672177903 shutdown_connections 2024-05-14T17:05:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51ab3f0 0x55c9e51b8a60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51bead0 0x55c9e51d6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 --2- 172.21.15.99:0/2672177903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51d6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 -- 172.21.15.99:0/2672177903 >> 172.21.15.99:0/2672177903 conn(0x55c9e51afe00 msgr2=0x55c9e51c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.090+0000 7ff802197740 1 -- 172.21.15.99:0/2672177903 shutdown_connections 2024-05-14T17:05:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 -- 172.21.15.99:0/2672177903 wait complete. 2024-05-14T17:05:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 -- start start 2024-05-14T17:05:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 Processor -- start 2024-05-14T17:05:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51c4e40 0x55c9e51c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81aa8f0 con 0x55c9e51bead0 2024-05-14T17:05:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81abd60 con 0x55c9e51c4e40 2024-05-14T17:05:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81aafc0 con 0x55c9e51b4210 2024-05-14T17:05:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.091+0000 7ff802197740 10 client.0 _ll_register_callbacks cb 0x7ff7f40107b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fc93f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51c68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:35256/0 (socket says 172.21.15.99:35256) 2024-05-14T17:05:28.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 -- 172.21.15.99:0/1675657308 learned_addr learned my addr 172.21.15.99:0/1675657308 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd941640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51c4e40 0x55c9e51c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 -- 172.21.15.99:0/1675657308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51c4e40 msgr2=0x55c9e51c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51c4e40 0x55c9e51c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 -- 172.21.15.99:0/1675657308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 msgr2=0x55c9e51c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 -- 172.21.15.99:0/1675657308 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e51c5900 con 0x55c9e51b4210 2024-05-14T17:05:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7fd140640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51c68a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7f401de50 tx=0x7ff7f401c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c40f4b0c7ff67ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:28.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/1675657308 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4026960 con 0x55c9e51b4210 2024-05-14T17:05:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/1675657308 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7ff7f4026b00 con 0x55c9e51b4210 2024-05-14T17:05:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.092+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f0036030 con 0x55c9e51b4210 2024-05-14T17:05:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/1675657308 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7f4026d00 con 0x55c9e51b4210 2024-05-14T17:05:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 10 client.25105 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c9e523a060 con 0x55c9e51b4210 2024-05-14T17:05:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/1675657308 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f0036030 con 0x55c9e51b4210 2024-05-14T17:05:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/1675657308 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff7f4025040 con 0x55c9e51b4210 2024-05-14T17:05:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 10 client.25105 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 10 client.25105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 20 client.25105 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e523a800 0x55c9e523cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 4 client.25105 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9e5122360 con 0x55c9e523a800 2024-05-14T17:05:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.093+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e523a800 0x55c9e523cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.094+0000 7ff7e0ff9640 10 client.25105 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.094+0000 7ff7fc93f640 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e523a800 0x55c9e523cc20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88128ec0d49732 server_cookie=cebce618ec5cb2c7 in_seq=0 out_seq=0 2024-05-14T17:05:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.097+0000 7ff7e0ff9640 1 -- 172.21.15.99:0/1675657308 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c9e5122360 con 0x55c9e523a800 2024-05-14T17:05:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.097+0000 7ff7e0ff9640 10 client.25105 handle_command_reply: tid=0 2024-05-14T17:05:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 client.25105 shutdown 2024-05-14T17:05:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 msgr2=0x55c9e51c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51c68a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7f401de50 tx=0x7ff7f401c870 comp rx=0 tx=0).stop 2024-05-14T17:05:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 shutdown_connections 2024-05-14T17:05:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c9e523a800 0x55c9e523cc20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c9e51c4e40 0x55c9e51c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9e51bead0 0x55c9e51c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 --2- 172.21.15.99:0/1675657308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9e51b4210 0x55c9e51c68a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 >> 172.21.15.99:0/1675657308 conn(0x55c9e51afe00 msgr2=0x55c9e51c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff7e0ff9640 0 client.25105 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 shutdown_connections 2024-05-14T17:05:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 1 -- 172.21.15.99:0/1675657308 wait complete. 2024-05-14T17:05:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.098+0000 7ff802197740 20 client.25105 trim_cache size 0 max 16384 2024-05-14T17:05:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:05:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:05:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:05:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff7e40b2a90 msgr2=0x7ff7e40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff7e40b2a90 0x7ff7e40b4f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55c9e51b4e90 tx=0x7ff7f400b390 comp rx=0 tx=0).stop 2024-05-14T17:05:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 msgr2=0x7ff7f81a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 0x7ff7f81a5010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005e30 tx=0x7ff7f0003ba0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 shutdown_connections 2024-05-14T17:05:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff7e40b2a90 0x7ff7e40b4f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8107940 0x7ff7f81a5010 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f8102340 0x7ff7f819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 --2- 172.21.15.99:0/535662401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f8101980 0x7ff7f8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 >> 172.21.15.99:0/535662401 conn(0x7ff7f8078ad0 msgr2=0x7ff7f8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 shutdown_connections 2024-05-14T17:05:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:26.099+0000 7ff7ff3a4640 1 -- 172.21.15.99:0/535662401 wait complete. 2024-05-14T17:05:29.406 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:26 smithi099 ceph-mon[31851]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:05:29.415 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:28 smithi099 ceph-mon[31851]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:05:29.424 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:26 smithi193 ceph-mon[35962]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:05:29.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:28 smithi193 ceph-mon[35962]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:05:29.443 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:26 smithi150 ceph-mon[36204]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:05:29.451 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:28 smithi150 ceph-mon[36204]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T17:05:31.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:30 smithi150 ceph-mon[36204]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:30 smithi099 ceph-mon[31851]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:30 smithi193 ceph-mon[35962]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:33.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:32 smithi150 ceph-mon[36204]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:32 smithi099 ceph-mon[31851]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:33.345 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:32 smithi193 ceph-mon[35962]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:35.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:34 smithi150 ceph-mon[36204]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:35.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:34 smithi099 ceph-mon[31851]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:34 smithi193 ceph-mon[35962]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:37.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:36 smithi150 ceph-mon[36204]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:37.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:36 smithi099 ceph-mon[31851]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:36 smithi193 ceph-mon[35962]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:39.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:38 smithi150 ceph-mon[36204]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 -- 172.21.15.99:0/1680967927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 msgr2=0x7f476810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 --2- 172.21.15.99:0/1680967927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f476810f930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4760009cd0 tx=0x7f476002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 -- 172.21.15.99:0/1680967927 shutdown_connections 2024-05-14T17:05:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 --2- 172.21.15.99:0/1680967927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f476810f930 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 --2- 172.21.15.99:0/1680967927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 0x7f4768102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 --2- 172.21.15.99:0/1680967927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 0x7f4768108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.234+0000 7f476daa7640 1 -- 172.21.15.99:0/1680967927 >> 172.21.15.99:0/1680967927 conn(0x7f47680fe550 msgr2=0x7f4768100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.235+0000 7f476daa7640 1 -- 172.21.15.99:0/1680967927 shutdown_connections 2024-05-14T17:05:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.235+0000 7f476daa7640 1 -- 172.21.15.99:0/1680967927 wait complete. 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.235+0000 7f476daa7640 1 Processor -- start 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 -- start start 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 0x7f4768196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f4768197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 0x7f476819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4768110b40 con 0x7f4768103420 2024-05-14T17:05:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47681109c0 con 0x7f4768102a60 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f476daa7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4768110cc0 con 0x7f4768108a70 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f4768197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f4768197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35704/0 (socket says 172.21.15.99:35704) 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 -- 172.21.15.99:0/1602607486 learned_addr learned my addr 172.21.15.99:0/1602607486 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 0x7f4768196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 -- 172.21.15.99:0/1602607486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 msgr2=0x7f476819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4767fff640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 0x7f476819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 0x7f476819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 -- 172.21.15.99:0/1602607486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 msgr2=0x7f4768196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 0x7f4768196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4766ffd640 1 -- 172.21.15.99:0/1602607486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476819bf80 con 0x7f4768103420 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 0x7f4768196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.236+0000 7f4767fff640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 0x7f476819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.237+0000 7f4766ffd640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f4768197520 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f475c00d950 tx=0x7f475c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.237+0000 7f4764ff9640 1 -- 172.21.15.99:0/1602607486 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c017030 con 0x7f4768103420 2024-05-14T17:05:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.237+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476819c270 con 0x7f4768103420 2024-05-14T17:05:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.237+0000 7f4764ff9640 1 -- 172.21.15.99:0/1602607486 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c004020 con 0x7f4768103420 2024-05-14T17:05:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.237+0000 7f4764ff9640 1 -- 172.21.15.99:0/1602607486 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c017030 con 0x7f4768103420 2024-05-14T17:05:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.237+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47681a3b00 con 0x7f4768103420 2024-05-14T17:05:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.240+0000 7f47707c8740 1 Processor -- start 2024-05-14T17:05:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.240+0000 7f47707c8740 1 -- start start 2024-05-14T17:05:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.241+0000 7f4764ff9640 1 -- 172.21.15.99:0/1602607486 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f475c005430 con 0x7f4768103420 2024-05-14T17:05:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47707c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b81d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47707c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b827da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f4764ff9640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f47500b2e00 0x7f47500b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47707c8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 0x56129b82a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47707c8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56129b787c30 con 0x56129b8282e0 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47707c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56129b7777d0 con 0x56129b81da20 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47707c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56129b77b810 con 0x56129b81d100 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f47500b2e00 0x7f47500b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f4764ff9640 1 -- 172.21.15.99:0/1602607486 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f475c018070 con 0x7f4768103420 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f4767fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 0x56129b82a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b827da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f4767fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 0x56129b82a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35720/0 (socket says 172.21.15.99:35720) 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.242+0000 7f4767fff640 1 -- 172.21.15.99:0/2670431219 learned_addr learned my addr 172.21.15.99:0/2670431219 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4766ffd640 1 -- 172.21.15.99:0/2670431219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 msgr2=0x56129b81d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4766ffd640 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b81d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4766ffd640 1 -- 172.21.15.99:0/2670431219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 msgr2=0x56129b82a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4766ffd640 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 0x56129b82a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f47500b2e00 0x7f47500b52c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f475800f4d0 tx=0x7f4758011040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4766ffd640 1 -- 172.21.15.99:0/2670431219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56129b48c210 con 0x56129b81da20 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4767fff640 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 0x56129b82a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f4766ffd640 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b827da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f475c0df2d0 tx=0x7f475c0a7420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd346847c2affb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f474dffb640 1 -- 172.21.15.99:0/2670431219 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c017070 con 0x56129b81da20 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f474dffb640 1 -- 172.21.15.99:0/2670431219 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c0a8970 con 0x56129b81da20 2024-05-14T17:05:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.243+0000 7f474dffb640 1 -- 172.21.15.99:0/2670431219 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c017070 con 0x56129b81da20 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 -- 172.21.15.99:0/2670431219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 msgr2=0x56129b827da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b827da0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f475c0df2d0 tx=0x7f475c0a7420 comp rx=0 tx=0).stop 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 -- 172.21.15.99:0/2670431219 shutdown_connections 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b8282e0 0x56129b82a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b827da0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 --2- 172.21.15.99:0/2670431219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b81d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 -- 172.21.15.99:0/2670431219 >> 172.21.15.99:0/2670431219 conn(0x56129b819610 msgr2=0x56129b8199f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 -- 172.21.15.99:0/2670431219 shutdown_connections 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 -- 172.21.15.99:0/2670431219 wait complete. 2024-05-14T17:05:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.244+0000 7f47707c8740 1 -- start start 2024-05-14T17:05:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.245+0000 7f47707c8740 1 Processor -- start 2024-05-14T17:05:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.245+0000 7f47707c8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b827d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.245+0000 7f47677fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b827d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.245+0000 7f47677fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b827d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35730/0 (socket says 172.21.15.99:35730) 2024-05-14T17:05:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.245+0000 7f47677fe640 1 -- 172.21.15.99:0/246472084 learned_addr learned my addr 172.21.15.99:0/246472084 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.245+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81da20 0x56129b825020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b825760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56129b45d790 con 0x56129b81d100 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56129b733890 con 0x56129b8282e0 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56129b82afb0 con 0x56129b81da20 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47707c8740 10 client.0 _ll_register_callbacks cb 0x56129b736d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81da20 0x56129b825020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4767fff640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b825760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 -- 172.21.15.99:0/246472084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 msgr2=0x56129b825760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b825760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 -- 172.21.15.99:0/246472084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 msgr2=0x56129b827d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b827d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56129b558620 con 0x56129b81da20 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4767fff640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b825760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47677fe640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b827d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4766ffd640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81da20 0x56129b825020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x56129b81e010 tx=0x7f475c0a7550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e2b96a986cd973 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4746ffd640 1 -- 172.21.15.99:0/246472084 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c017070 con 0x56129b81da20 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f4746ffd640 1 -- 172.21.15.99:0/246472084 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f475c0a5450 con 0x56129b81da20 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.246+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c0a8970 con 0x56129b81da20 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f4746ffd640 1 -- 172.21.15.99:0/246472084 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f475c0ad210 con 0x56129b81da20 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 10 client.25111 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56129b836ab0 con 0x56129b81da20 2024-05-14T17:05:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f4746ffd640 1 -- 172.21.15.99:0/246472084 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c0a8970 con 0x56129b81da20 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f4746ffd640 1 -- 172.21.15.99:0/246472084 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f475c0a50f0 con 0x56129b81da20 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 10 client.25111 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 10 client.25111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 20 client.25111 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b83a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 4 client.25111 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.247+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56129b789b80 con 0x56129b837e00 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.248+0000 7f47677fe640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b83a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.248+0000 7f4746ffd640 10 client.25111 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.248+0000 7f47677fe640 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b83a1f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff54d23036448db server_cookie=4638d42b61022e0 in_seq=0 out_seq=0 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f4746ffd640 1 -- 172.21.15.99:0/246472084 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56129b789b80 con 0x56129b837e00 2024-05-14T17:05:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f4746ffd640 10 client.25111 handle_command_reply: tid=0 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 client.25111 shutdown 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81da20 msgr2=0x56129b825020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81da20 0x56129b825020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x56129b81e010 tx=0x7f475c0a7550 comp rx=0 tx=0).stop 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 shutdown_connections 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b83a1f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f4746ffd640 0 client.25111 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b825760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81da20 0x56129b825020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 --2- 172.21.15.99:0/246472084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b827d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 >> 172.21.15.99:0/246472084 conn(0x56129b819610 msgr2=0x56129b81eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 shutdown_connections 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 1 -- 172.21.15.99:0/246472084 wait complete. 2024-05-14T17:05:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.249+0000 7f47707c8740 20 client.25111 trim_cache size 0 max 16384 2024-05-14T17:05:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:38 smithi099 ceph-mon[31851]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 Processor -- start 2024-05-14T17:05:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 -- start start 2024-05-14T17:05:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b810100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b81f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b8282e0 0x56129b827860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47681aed40 con 0x56129b81d100 2024-05-14T17:05:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47681ac610 con 0x56129b81da20 2024-05-14T17:05:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47707c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47681aa460 con 0x56129b8282e0 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b810100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b810100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35734/0 (socket says 172.21.15.99:35734) 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 -- 172.21.15.99:0/1475956866 learned_addr learned my addr 172.21.15.99:0/1475956866 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f4767fff640 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b8282e0 0x56129b827860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f4766ffd640 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b81f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 -- 172.21.15.99:0/1475956866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b8282e0 msgr2=0x56129b827860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b8282e0 0x56129b827860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 -- 172.21.15.99:0/1475956866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 msgr2=0x56129b81f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b81f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 -- 172.21.15.99:0/1475956866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56129b825ad0 con 0x56129b81d100 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.265+0000 7f47677fe640 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b810100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f475801c360 tx=0x7f4758013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36d95af714b690 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f474d7fa640 1 -- 172.21.15.99:0/1475956866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475802e070 con 0x56129b81d100 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f474d7fa640 1 -- 172.21.15.99:0/1475956866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475801b8c0 con 0x56129b81d100 2024-05-14T17:05:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f474d7fa640 1 -- 172.21.15.99:0/1475956866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758027800 con 0x56129b81d100 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 -- 172.21.15.99:0/1475956866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 msgr2=0x56129b810100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b810100 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f475801c360 tx=0x7f4758013000 comp rx=0 tx=0).stop 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 -- 172.21.15.99:0/1475956866 shutdown_connections 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b8282e0 0x56129b827860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b81da20 0x56129b81f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 --2- 172.21.15.99:0/1475956866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81d100 0x56129b810100 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.266+0000 7f47707c8740 1 -- 172.21.15.99:0/1475956866 >> 172.21.15.99:0/1475956866 conn(0x56129b819610 msgr2=0x56129b81eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 -- 172.21.15.99:0/1475956866 shutdown_connections 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 -- 172.21.15.99:0/1475956866 wait complete. 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 -- start start 2024-05-14T17:05:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 Processor -- start 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b83af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81da20 0x56129b83b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f47707c8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b83b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.267+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81da20 0x56129b83b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47707c8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47681aaad0 con 0x56129b81da20 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81da20 0x56129b83b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35736/0 (socket says 172.21.15.99:35736) 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f4766ffd640 1 -- 172.21.15.99:0/1602601668 learned_addr learned my addr 172.21.15.99:0/1602601668 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47681abf40 con 0x56129b8282e0 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47681ab1a0 con 0x56129b81d100 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b83af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47707c8740 10 client.0 _ll_register_callbacks cb 0x7f475c0a7370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f4767fff640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b83b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 -- 172.21.15.99:0/1602601668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 msgr2=0x56129b83b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b83b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 -- 172.21.15.99:0/1602601668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81da20 msgr2=0x56129b83b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81da20 0x56129b83b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 -- 172.21.15.99:0/1602601668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56129b8a1420 con 0x56129b81d100 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47677fe640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b83af30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4758011010 tx=0x7f475801b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56b8a95f90398a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f4746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f4746ffd640 1 -- 172.21.15.99:0/1602601668 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758036020 con 0x56129b81d100 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f4746ffd640 1 -- 172.21.15.99:0/1602601668 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f475801ba70 con 0x56129b81d100 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.268+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475801b8c0 con 0x56129b81d100 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f4746ffd640 1 -- 172.21.15.99:0/1602601668 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4758041050 con 0x56129b81d100 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 10 client.25117 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56129b8a2710 con 0x56129b81d100 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f4746ffd640 1 -- 172.21.15.99:0/1602601668 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475801b8c0 con 0x56129b81d100 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f4746ffd640 1 -- 172.21.15.99:0/1602601668 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4758031030 con 0x56129b81d100 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 10 client.25117 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 10 client.25117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 20 client.25117 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b8a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 4 client.25117 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56129b783360 con 0x56129b837e00 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.269+0000 7f4766ffd640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b8a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.270+0000 7f4746ffd640 10 client.25117 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.270+0000 7f4766ffd640 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b8a4db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31b5099a02793f6 server_cookie=40de35ee75d5a9ee in_seq=0 out_seq=0 2024-05-14T17:05:39.276 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.273+0000 7f4746ffd640 1 -- 172.21.15.99:0/1602601668 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56129b783360 con 0x56129b837e00 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.273+0000 7f4746ffd640 10 client.25117 handle_command_reply: tid=0 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.273+0000 7f47707c8740 1 client.25117 shutdown 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 msgr2=0x56129b83af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b83af30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4758011010 tx=0x7f475801b410 comp rx=0 tx=0).stop 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 shutdown_connections 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56129b837e00 0x56129b8a4db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56129b8282e0 0x56129b83b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56129b81da20 0x56129b83b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f4746ffd640 0 client.25117 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 --2- 172.21.15.99:0/1602601668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56129b81d100 0x56129b83af30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 >> 172.21.15.99:0/1602601668 conn(0x56129b819610 msgr2=0x56129b81f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 shutdown_connections 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 1 -- 172.21.15.99:0/1602601668 wait complete. 2024-05-14T17:05:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f47707c8740 20 client.25117 trim_cache size 0 max 16384 2024-05-14T17:05:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:05:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:05:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:05:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f47500b2e00 msgr2=0x7f47500b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f476daa7640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f47500b2e00 0x7f47500b52c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f475800f4d0 tx=0x7f4758011040 comp rx=0 tx=0).stop 2024-05-14T17:05:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 msgr2=0x7f4768197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.274+0000 7f476daa7640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f4768197520 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f475c00d950 tx=0x7f475c00de10 comp rx=0 tx=0).stop 2024-05-14T17:05:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 shutdown_connections 2024-05-14T17:05:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f47500b2e00 0x7f47500b52c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4768108a70 0x7f476819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4768103420 0x7f4768197520 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 --2- 172.21.15.99:0/1602607486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768102a60 0x7f4768196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 >> 172.21.15.99:0/1602607486 conn(0x7f47680fe550 msgr2=0x7f47680fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 shutdown_connections 2024-05-14T17:05:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:39.275+0000 7f476daa7640 1 -- 172.21.15.99:0/1602607486 wait complete. 2024-05-14T17:05:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:38 smithi193 ceph-mon[35962]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:41.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:40 smithi150 ceph-mon[36204]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:40 smithi099 ceph-mon[31851]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:40 smithi193 ceph-mon[35962]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:43.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:42 smithi150 ceph-mon[36204]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T17:05:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:42 smithi099 ceph-mon[31851]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T17:05:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:42 smithi193 ceph-mon[35962]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T17:05:45.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:44 smithi150 ceph-mon[36204]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:44 smithi099 ceph-mon[31851]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:44 smithi193 ceph-mon[35962]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:05:47.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:46 smithi150 ceph-mon[36204]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:47.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:46 smithi099 ceph-mon[31851]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:46 smithi193 ceph-mon[35962]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr:Note: switching to '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'. 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr:state without impacting any branches by switching back to a branch. 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-14T17:05:47.381 INFO:tasks.workunit.client.0.smithi099.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: git switch -c 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr:Or undo this operation with: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: git switch - 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-05-14T17:05:47.382 INFO:tasks.workunit.client.0.smithi099.stderr:HEAD is now at 9b63866860f qa/quiescer: relax some timing requirements in the quiescer 2024-05-14T17:05:47.388 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-14T17:05:47.414 INFO:tasks.workunit.client.0.smithi099.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-14T17:05:47.416 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-14T17:05:47.416 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-14T17:05:48.143 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-14T17:05:48.197 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-14T17:05:48.238 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-14T17:05:48.239 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-14T17:05:48.239 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-14T17:05:48.278 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-14T17:05:48.281 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T17:05:48.281 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-14T17:05:48.338 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-14T17:05:48.339 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-14T17:05:48.340 DEBUG:teuthology.orchestra.run.smithi099:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-14T17:05:48.401 INFO:tasks.workunit.client.0.smithi099.stderr:+ dbench 1 2024-05-14T17:05:48.410 INFO:tasks.workunit.client.0.smithi099.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-14T17:05:48.410 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-05-14T17:05:48.410 INFO:tasks.workunit.client.0.smithi099.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-14T17:05:48.415 INFO:tasks.workunit.client.0.smithi099.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-14T17:05:48.515 INFO:tasks.workunit.client.0.smithi099.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-14T17:05:48.516 INFO:tasks.workunit.client.0.smithi099.stdout:releasing clients 2024-05-14T17:05:49.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:48 smithi150 ceph-mon[36204]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:49.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:48 smithi099 ceph-mon[31851]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:48 smithi193 ceph-mon[35962]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:05:49.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 3390 35.49 MB/sec warmup 1 sec latency 6.649 ms 2024-05-14T17:05:50.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:49 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:50.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:49 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:50.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:49 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:50.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 6026 23.87 MB/sec warmup 2 sec latency 12.900 ms 2024-05-14T17:05:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:50 smithi150 ceph-mon[36204]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-05-14T17:05:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:50 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:50 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:50 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:05:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:50 smithi099 ceph-mon[31851]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-05-14T17:05:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:50 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:50 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:50 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:05:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:50 smithi193 ceph-mon[35962]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-05-14T17:05:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:50 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:50 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:50 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:05:51.400 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T17:05:51.400 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.7 2024-05-14T17:05:51.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:05:51.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 8794 21.49 MB/sec warmup 3 sec latency 11.709 ms 2024-05-14T17:05:51.649 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1753156770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4107940 msgr2=0x7fdcc4107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1753156770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4107940 0x7fdcc4107d20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009930 tx=0x7fdcb402a420 comp rx=0 tx=0).stop 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1753156770 shutdown_connections 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1753156770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4102340 0x7fdcc410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1753156770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcc4101980 0x7fdcc4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1753156770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4107940 0x7fdcc4107d20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1753156770 >> 172.21.15.99:0/1753156770 conn(0x7fdcc4078ad0 msgr2=0x7fdcc4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1753156770 shutdown_connections 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1753156770 wait complete. 2024-05-14T17:05:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.364+0000 7fdcc9cc8640 1 Processor -- start 2024-05-14T17:05:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 -- start start 2024-05-14T17:05:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcc4101980 0x7fdcc419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 0x7fdcc419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4107940 0x7fdcc41a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111c00 con 0x7fdcc4102340 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111a80 con 0x7fdcc4107940 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc9cc8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111d80 con 0x7fdcc4101980 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 0x7fdcc419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 0x7fdcc419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52512/0 (socket says 172.21.15.99:52512) 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1546561751 learned_addr learned my addr 172.21.15.99:0/1546561751 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcc4101980 0x7fdcc419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4107940 0x7fdcc41a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1546561751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcc4101980 msgr2=0x7fdcc419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcc4101980 0x7fdcc419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1546561751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4107940 msgr2=0x7fdcc41a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4107940 0x7fdcc41a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1546561751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc41a5660 con 0x7fdcc4102340 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4107940 0x7fdcc41a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.365+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 0x7fdcc419a5e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800e9f0 tx=0x7fdcb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.366+0000 7fdcc0ff9640 1 -- 172.21.15.99:0/1546561751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800cd80 con 0x7fdcc4102340 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.366+0000 7fdcc0ff9640 1 -- 172.21.15.99:0/1546561751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb80054d0 con 0x7fdcc4102340 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.366+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41a5830 con 0x7fdcc4102340 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.366+0000 7fdcc0ff9640 1 -- 172.21.15.99:0/1546561751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8010690 con 0x7fdcc4102340 2024-05-14T17:05:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.366+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a5d50 con 0x7fdcc4102340 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.368+0000 7fdcc0ff9640 1 -- 172.21.15.99:0/1546561751 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdcb8004e90 con 0x7fdcc4102340 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.368+0000 7fdccc9e4740 1 Processor -- start 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.368+0000 7fdccc9e4740 1 -- start start 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc0ff9640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdcac0b2a90 0x7fdcac0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 0x55bb242abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242bd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb2421ec30 con 0x55bb242ab7f0 2024-05-14T17:05:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bb2420e7d0 con 0x55bb242baee0 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bb24212810 con 0x55bb242ac110 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdcac0b2a90 0x7fdcac0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 0x55bb242abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 0x55bb242abbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52516/0 (socket says 172.21.15.99:52516) 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 -- 172.21.15.99:0/1071180923 learned_addr learned my addr 172.21.15.99:0/1071180923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc0ff9640 1 -- 172.21.15.99:0/1546561751 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdcb80e2a10 con 0x7fdcc4102340 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242bd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 -- 172.21.15.99:0/1071180923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 msgr2=0x55bb242ba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242ba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 -- 172.21.15.99:0/1071180923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 msgr2=0x55bb242bd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242bd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 -- 172.21.15.99:0/1071180923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23f23210 con 0x55bb242ab7f0 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242ba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdcac0b2a90 0x7fdcac0b4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005fd0 tx=0x7fdcbc012b30 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.369+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 0x55bb242abbd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdcb402a940 tx=0x7fdcb4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373221091b65e384 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdca9ffb640 1 -- 172.21.15.99:0/1071180923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4040070 con 0x55bb242ab7f0 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdca9ffb640 1 -- 172.21.15.99:0/1071180923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb4004030 con 0x55bb242ab7f0 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdca9ffb640 1 -- 172.21.15.99:0/1071180923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4035040 con 0x55bb242ab7f0 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1071180923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 msgr2=0x55bb242abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 0x55bb242abbd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdcb402a940 tx=0x7fdcb4005be0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1071180923 shutdown_connections 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242bd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242ba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1071180923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ab7f0 0x55bb242abbd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1071180923 >> 172.21.15.99:0/1071180923 conn(0x55bb242a7e00 msgr2=0x55bb242a81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1071180923 shutdown_connections 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.370+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1071180923 wait complete. 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 -- start start 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 Processor -- start 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb23ef4790 con 0x55bb242ac110 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bb241ca890 con 0x55bb242baee0 2024-05-14T17:05:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bb242bdb50 con 0x55bb242ab7f0 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdccc9e4740 10 client.0 _ll_register_callbacks cb 0x55bb241cdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242b7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52530/0 (socket says 172.21.15.99:52530) 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/547367700 learned_addr learned my addr 172.21.15.99:0/547367700 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.371+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc37fe640 1 -- 172.21.15.99:0/547367700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 msgr2=0x55bb242b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc37fe640 1 -- 172.21.15.99:0/547367700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 msgr2=0x55bb242b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc37fe640 1 -- 172.21.15.99:0/547367700 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb23fef620 con 0x55bb242ab7f0 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242ba7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1650941c2f15be48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdca2ffd640 1 -- 172.21.15.99:0/547367700 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb404c020 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb4004030 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdca2ffd640 1 -- 172.21.15.99:0/547367700 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdcb40051c0 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdca2ffd640 1 -- 172.21.15.99:0/547367700 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcb4057050 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdccc9e4740 10 client.25129 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb242c9570 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.372+0000 7fdca2ffd640 1 -- 172.21.15.99:0/547367700 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb4004030 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdca2ffd640 1 -- 172.21.15.99:0/547367700 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdcb40035f0 con 0x55bb242ab7f0 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdccc9e4740 10 client.25129 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdccc9e4740 10 client.25129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdccc9e4740 20 client.25129 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb242ca8c0 0x55bb242cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdccc9e4740 4 client.25129 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb24220b80 con 0x55bb242ca8c0 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb242ca8c0 0x55bb242cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdca2ffd640 10 client.25129 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.373+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb242ca8c0 0x55bb242cccb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a28d0f6f2bc2f7a server_cookie=82cb62dbe263bb0 in_seq=0 out_seq=0 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdca2ffd640 1 -- 172.21.15.99:0/547367700 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb24220b80 con 0x55bb242ca8c0 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdca2ffd640 10 client.25129 handle_command_reply: tid=0 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 client.25129 shutdown 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 msgr2=0x55bb242ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242ba7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4004410 comp rx=0 tx=0).stop 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 shutdown_connections 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb242ca8c0 0x55bb242cccb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/547367700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242ba7b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdca2ffd640 0 client.25129 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 >> 172.21.15.99:0/547367700 conn(0x55bb242a7e00 msgr2=0x55bb242adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 shutdown_connections 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 1 -- 172.21.15.99:0/547367700 wait complete. 2024-05-14T17:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.374+0000 7fdccc9e4740 20 client.25129 trim_cache size 0 max 16384 2024-05-14T17:05:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 Processor -- start 2024-05-14T17:05:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 -- start start 2024-05-14T17:05:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41aec10 con 0x55bb242ac110 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ac4e0 con 0x55bb242baee0 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41aa330 con 0x55bb242ab7f0 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52534/0 (socket says 172.21.15.99:52534) 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1158841625 learned_addr learned my addr 172.21.15.99:0/1158841625 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1158841625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 msgr2=0x55bb242b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1158841625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 msgr2=0x55bb242adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc2ffd640 1 -- 172.21.15.99:0/1158841625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb242b6a50 con 0x55bb242ac110 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.390+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242a2f90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c6a0 tx=0x7fdcb801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276b0e41ad3e4f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdca97fa640 1 -- 172.21.15.99:0/1158841625 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80aa420 con 0x55bb242ac110 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdca97fa640 1 -- 172.21.15.99:0/1158841625 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb806fa70 con 0x55bb242ac110 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdca97fa640 1 -- 172.21.15.99:0/1158841625 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80a9f80 con 0x55bb242ac110 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1158841625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 msgr2=0x55bb242a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242a2f90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c6a0 tx=0x7fdcb801c3b0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1158841625 shutdown_connections 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242baee0 0x55bb242adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242ac110 0x55bb242a2f90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/1158841625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ab7f0 0x55bb242b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1158841625 >> 172.21.15.99:0/1158841625 conn(0x55bb242a7e00 msgr2=0x55bb242ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1158841625 shutdown_connections 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.391+0000 7fdccc9e4740 1 -- 172.21.15.99:0/1158841625 wait complete. 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 -- start start 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 Processor -- start 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242ab7f0 0x55bb242c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242be700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 0x55bb242bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41aa9a0 con 0x55bb242baee0 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41abe10 con 0x55bb242ab7f0 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ab070 con 0x55bb242ac110 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdccc9e4740 10 client.0 _ll_register_callbacks cb 0x7fdcb402aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdcc3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 0x55bb242bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdcc3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 0x55bb242bec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52540/0 (socket says 172.21.15.99:52540) 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdcc3fff640 1 -- 172.21.15.99:0/2404070006 learned_addr learned my addr 172.21.15.99:0/2404070006 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdcc2ffd640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242be700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.392+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242ab7f0 0x55bb242c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdcc3fff640 1 -- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 msgr2=0x55bb242be700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242be700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdcc3fff640 1 -- 172.21.15.99:0/2404070006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242ab7f0 msgr2=0x55bb242c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242ab7f0 0x55bb242c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdcc3fff640 1 -- 172.21.15.99:0/2404070006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb242abf40 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 0x55bb242bec80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005c40 tx=0x7fdcbc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad7abc13f3e2538 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdca2ffd640 1 -- 172.21.15.99:0/2404070006 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02e060 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdca2ffd640 1 -- 172.21.15.99:0/2404070006 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdcbc00f810 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb806fa70 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdca2ffd640 1 -- 172.21.15.99:0/2404070006 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc00fa10 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb24334da0 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdca2ffd640 1 -- 172.21.15.99:0/2404070006 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb806fa70 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdca2ffd640 1 -- 172.21.15.99:0/2404070006 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdcbc028070 con 0x55bb242baee0 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 20 client.15549 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb24335550 0x55bb24337970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 4 client.15549 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.393+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb2421a360 con 0x55bb24335550 2024-05-14T17:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.394+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb24335550 0x55bb24337970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.394+0000 7fdcc37fe640 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb24335550 0x55bb24337970 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f79f50ba1a8791 server_cookie=f81852f2a97e23dd in_seq=0 out_seq=0 2024-05-14T17:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.394+0000 7fdca2ffd640 10 client.15549 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:52.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 11334 19.63 MB/sec warmup 4 sec latency 31.700 ms 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdca2ffd640 1 -- 172.21.15.99:0/2404070006 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb2421a360 con 0x55bb24335550 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdca2ffd640 10 client.15549 handle_command_reply: tid=0 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 client.15549 shutdown 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 msgr2=0x55bb242bec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 0x55bb242bec80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005c40 tx=0x7fdcbc01a000 comp rx=0 tx=0).stop 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 shutdown_connections 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb24335550 0x55bb24337970 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb242baee0 0x55bb242bec80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb242ac110 0x55bb242be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 --2- 172.21.15.99:0/2404070006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb242ab7f0 0x55bb242c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdca2ffd640 0 client.15549 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 >> 172.21.15.99:0/2404070006 conn(0x55bb242a7e00 msgr2=0x55bb242a7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 shutdown_connections 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 1 -- 172.21.15.99:0/2404070006 wait complete. 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.529+0000 7fdccc9e4740 20 client.15549 trim_cache size 0 max 16384 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:05:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdcac0b2a90 msgr2=0x7fdcac0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdcac0b2a90 0x7fdcac0b4f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005fd0 tx=0x7fdcbc012b30 comp rx=0 tx=0).stop 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 msgr2=0x7fdcc419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 0x7fdcc419a5e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800e9f0 tx=0x7fdcb800edf0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 shutdown_connections 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdcac0b2a90 0x7fdcac0b4f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc4107940 0x7fdcc41a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc4102340 0x7fdcc419a5e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 --2- 172.21.15.99:0/1546561751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcc4101980 0x7fdcc419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 >> 172.21.15.99:0/1546561751 conn(0x7fdcc4078ad0 msgr2=0x7fdcc41045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 shutdown_connections 2024-05-14T17:05:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:05:52.530+0000 7fdcc9cc8640 1 -- 172.21.15.99:0/1546561751 wait complete. 2024-05-14T17:05:53.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:52 smithi099 ceph-mon[31851]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-05-14T17:05:53.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:52 smithi150 ceph-mon[36204]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-05-14T17:05:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:52 smithi193 ceph-mon[35962]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-05-14T17:05:53.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 -- 172.21.15.99:0/2607699191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102df0 msgr2=0x7fa21810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2607699191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102df0 0x7fa21810f2e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b0a0 tx=0x7fa21000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 -- 172.21.15.99:0/2607699191 shutdown_connections 2024-05-14T17:05:53.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2607699191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102df0 0x7fa21810f2e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2607699191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218108510 0x7fa2181088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2607699191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa218102430 0x7fa2181028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 -- 172.21.15.99:0/2607699191 >> 172.21.15.99:0/2607699191 conn(0x7fa2180793b0 msgr2=0x7fa2180797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 -- 172.21.15.99:0/2607699191 shutdown_connections 2024-05-14T17:05:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.305+0000 7fa21cd83700 1 -- 172.21.15.99:0/2607699191 wait complete. 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 Processor -- start 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 -- start start 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 0x7fa21819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218102df0 0x7fa21819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa218108510 0x7fa218198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2181125e0 con 0x7fa218102430 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa218112460 con 0x7fa218102df0 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21cd83700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa218112760 con 0x7fa218108510 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 0x7fa21819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 0x7fa21819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52564/0 (socket says 172.21.15.99:52564) 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa215d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218102df0 0x7fa21819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa21659c700 1 -- 172.21.15.99:0/2296095226 learned_addr learned my addr 172.21.15.99:0/2296095226 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.306+0000 7fa216d9d700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa218108510 0x7fa218198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa215d9b700 1 -- 172.21.15.99:0/2296095226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa218108510 msgr2=0x7fa218198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa215d9b700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa218108510 0x7fa218198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa215d9b700 1 -- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 msgr2=0x7fa21819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa215d9b700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 0x7fa21819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa215d9b700 1 -- 172.21.15.99:0/2296095226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa218198fe0 con 0x7fa218102df0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa21659c700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 0x7fa21819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa215d9b700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218102df0 0x7fa21819ec00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00e9f0 tx=0x7fa20c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004220 con 0x7fa218102df0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2181992d0 con 0x7fa218102df0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa20c004ce0 con 0x7fa218102df0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c01c070 con 0x7fa218102df0 2024-05-14T17:05:53.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.307+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2181a5590 con 0x7fa218102df0 2024-05-14T17:05:53.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.308+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21806a230 con 0x7fa218102df0 2024-05-14T17:05:53.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.310+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa20c004750 con 0x7fa218102df0 2024-05-14T17:05:53.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.310+0000 7fa2077fe700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2000b29f0 0x7fa2000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:53.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.310+0000 7fa21659c700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2000b29f0 0x7fa2000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:53.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.310+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa20c0e2560 con 0x7fa218102df0 2024-05-14T17:05:53.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.311+0000 7fa21659c700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2000b29f0 0x7fa2000b4eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2181999c0 tx=0x7fa208009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:53.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.314+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa20c0aa000 con 0x7fa218102df0 2024-05-14T17:05:53.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 14507 19.43 MB/sec warmup 5 sec latency 40.595 ms 2024-05-14T17:05:53.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.588+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa218199de0 con 0x7fa218102df0 2024-05-14T17:05:53.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.588+0000 7fa2077fe700 1 -- 172.21.15.99:0/2296095226 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7fa218199de0 con 0x7fa218102df0 2024-05-14T17:05:53.589 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:05:53.589 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:05:53.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2000b29f0 msgr2=0x7fa2000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2000b29f0 0x7fa2000b4eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2181999c0 tx=0x7fa208009260 comp rx=0 tx=0).stop 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218102df0 msgr2=0x7fa21819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218102df0 0x7fa21819ec00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00e9f0 tx=0x7fa20c00ed00 comp rx=0 tx=0).stop 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 shutdown_connections 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2000b29f0 0x7fa2000b4eb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa218108510 0x7fa218198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa218102430 0x7fa21819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 --2- 172.21.15.99:0/2296095226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa218102df0 0x7fa21819ec00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 >> 172.21.15.99:0/2296095226 conn(0x7fa2180793b0 msgr2=0x7fa218100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 shutdown_connections 2024-05-14T17:05:53.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:53.591+0000 7fa21cd83700 1 -- 172.21.15.99:0/2296095226 wait complete. 2024-05-14T17:05:53.593 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:05:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:53 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2296095226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:53 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2296095226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:53 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2296095226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:54.337 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0' 2024-05-14T17:05:54.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0 2024-05-14T17:05:54.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 17723 19.14 MB/sec warmup 6 sec latency 12.361 ms 2024-05-14T17:05:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:55.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:54 smithi099 ceph-mon[31851]: pgmap v71: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 729 KiB/s wr, 137 op/s 2024-05-14T17:05:55.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:54 smithi150 ceph-mon[36204]: pgmap v71: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 729 KiB/s wr, 137 op/s 2024-05-14T17:05:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:54 smithi193 ceph-mon[35962]: pgmap v71: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 729 KiB/s wr, 137 op/s 2024-05-14T17:05:55.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 20342 17.79 MB/sec warmup 7 sec latency 18.838 ms 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 -- 172.21.15.99:0/2494722415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf98108620 msgr2=0x7faf98108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 --2- 172.21.15.99:0/2494722415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf98108620 0x7faf98108a00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 -- 172.21.15.99:0/2494722415 shutdown_connections 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 --2- 172.21.15.99:0/2494722415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf981028c0 0x7faf98110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 --2- 172.21.15.99:0/2494722415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf98108620 0x7faf98108a00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 --2- 172.21.15.99:0/2494722415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf98101f00 0x7faf98102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.146+0000 7fafa00ae700 1 -- 172.21.15.99:0/2494722415 >> 172.21.15.99:0/2494722415 conn(0x7faf98077780 msgr2=0x7faf98077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 -- 172.21.15.99:0/2494722415 shutdown_connections 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 -- 172.21.15.99:0/2494722415 wait complete. 2024-05-14T17:05:56.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 Processor -- start 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 -- start start 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf98101f00 0x7faf9819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 0x7faf9819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf98108620 0x7faf98199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf98113770 con 0x7faf981028c0 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf981135f0 con 0x7faf98101f00 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7fafa00ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf981138f0 con 0x7faf98108620 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.147+0000 7faf9d649700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 0x7faf9819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9de4a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf98101f00 0x7faf9819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 0x7faf9819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43320/0 (socket says 172.21.15.99:43320) 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 -- 172.21.15.99:0/4045878827 learned_addr learned my addr 172.21.15.99:0/4045878827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9e64b700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf98108620 0x7faf98199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 -- 172.21.15.99:0/4045878827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf98108620 msgr2=0x7faf98199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf98108620 0x7faf98199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 -- 172.21.15.99:0/4045878827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf98101f00 msgr2=0x7faf9819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf98101f00 0x7faf9819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 -- 172.21.15.99:0/4045878827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9819a070 con 0x7faf981028c0 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf9d649700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 0x7faf9819fc90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faf8800e9c0 tx=0x7faf8800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf8effd700 1 -- 172.21.15.99:0/4045878827 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8800cbd0 con 0x7faf981028c0 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf8effd700 1 -- 172.21.15.99:0/4045878827 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf8800ee60 con 0x7faf981028c0 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7faf8effd700 1 -- 172.21.15.99:0/4045878827 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880055c0 con 0x7faf981028c0 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9819a360 con 0x7faf981028c0 2024-05-14T17:05:56.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.148+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a6620 con 0x7faf981028c0 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.151+0000 7faf8effd700 1 -- 172.21.15.99:0/4045878827 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faf88004750 con 0x7faf981028c0 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.151+0000 7faf8effd700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faf840b2b20 0x7faf840b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.151+0000 7faf9de4a700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faf840b2b20 0x7faf840b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.152+0000 7faf8effd700 1 -- 172.21.15.99:0/4045878827 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7faf880e27f0 con 0x7faf981028c0 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.152+0000 7faf9de4a700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faf840b2b20 0x7faf840b4fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf94006010 tx=0x7faf9401a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.152+0000 7fafaa944b80 1 Processor -- start 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.152+0000 7fafaa944b80 1 -- start start 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.152+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 0x55766214a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.152+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214abc0 0x557662144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7fafaa944b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55766209ee20 con 0x55766214a2a0 2024-05-14T17:05:56.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7fafaa944b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55766208e9c0 con 0x55766214abc0 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7fafaa944b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557662092a00 con 0x557662144f70 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 0x55766214a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 0x55766214a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43334/0 (socket says 172.21.15.99:43334) 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 -- 172.21.15.99:0/4063218003 learned_addr learned my addr 172.21.15.99:0/4063218003 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9e64b700 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9d649700 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214abc0 0x557662144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 -- 172.21.15.99:0/4063218003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 msgr2=0x5576621453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 -- 172.21.15.99:0/4063218003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214abc0 msgr2=0x557662144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214abc0 0x557662144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 -- 172.21.15.99:0/4063218003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557661fdea60 con 0x55766214a2a0 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9e64b700 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:56.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf9de4a700 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 0x55766214a680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faf98078ce0 tx=0x7faf9401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02252b97a4397fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf83fff700 1 -- 172.21.15.99:0/4063218003 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9400bc00 con 0x55766214a2a0 2024-05-14T17:05:56.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf83fff700 1 -- 172.21.15.99:0/4063218003 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9400bda0 con 0x55766214a2a0 2024-05-14T17:05:56.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.153+0000 7faf83fff700 1 -- 172.21.15.99:0/4063218003 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9400f9a0 con 0x55766214a2a0 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 -- 172.21.15.99:0/4063218003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 msgr2=0x55766214a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 0x55766214a680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faf98078ce0 tx=0x7faf9401f000 comp rx=0 tx=0).stop 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 -- 172.21.15.99:0/4063218003 shutdown_connections 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214a2a0 0x55766214a680 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 --2- 172.21.15.99:0/4063218003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214abc0 0x557662144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 -- 172.21.15.99:0/4063218003 >> 172.21.15.99:0/4063218003 conn(0x55766206b3e0 msgr2=0x55766213ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 -- 172.21.15.99:0/4063218003 shutdown_connections 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.154+0000 7fafaa944b80 1 -- 172.21.15.99:0/4063218003 wait complete. 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 -- start start 2024-05-14T17:05:56.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 Processor -- start 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557662144f70 0x5576621508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55766214a2a0 0x55766214dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x55766214e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557661f95130 con 0x55766214abc0 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557662030950 con 0x557662144f70 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557662151560 con 0x55766214a2a0 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7fafaa944b80 10 client.0 _ll_register_callbacks cb 0x557662025710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7faf9e64b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x55766214e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7faf9e64b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x55766214e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43346/0 (socket says 172.21.15.99:43346) 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7faf9d649700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55766214a2a0 0x55766214dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7faf9e64b700 1 -- 172.21.15.99:0/34869193 learned_addr learned my addr 172.21.15.99:0/34869193 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.155+0000 7faf9de4a700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557662144f70 0x5576621508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9d649700 1 -- 172.21.15.99:0/34869193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557662144f70 msgr2=0x5576621508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9d649700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557662144f70 0x5576621508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9d649700 1 -- 172.21.15.99:0/34869193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 msgr2=0x55766214e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9d649700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x55766214e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9d649700 1 -- 172.21.15.99:0/34869193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766202d820 con 0x55766214a2a0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9e64b700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x55766214e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf9d649700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55766214a2a0 0x55766214dc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf880dff70 tx=0x7faf880aa060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3bdc339f0019f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf817fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf817fa700 1 -- 172.21.15.99:0/34869193 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880b22a0 con 0x55766214a2a0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf817fa700 1 -- 172.21.15.99:0/34869193 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7faf880a60f0 con 0x55766214a2a0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9400bda0 con 0x55766214a2a0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf817fa700 1 -- 172.21.15.99:0/34869193 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf880a9610 con 0x55766214a2a0 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7fafaa944b80 10 client.25153 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55766215bd60 con 0x55766214a2a0 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf817fa700 1 -- 172.21.15.99:0/34869193 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9400bda0 con 0x55766214a2a0 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7faf817fa700 1 -- 172.21.15.99:0/34869193 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faf88004a40 con 0x55766214a2a0 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.156+0000 7fafaa944b80 10 client.25153 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7fafaa944b80 10 client.25153 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7fafaa944b80 20 client.25153 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7fafaa944b80 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55766215e2f0 0x5576621606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7fafaa944b80 4 client.25153 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55766210c4f0 con 0x55766215e2f0 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7faf9de4a700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55766215e2f0 0x5576621606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7faf9de4a700 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55766215e2f0 0x5576621606e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2647f3ce628f8e1c server_cookie=5f770aa902df3a05 in_seq=0 out_seq=0 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.157+0000 7faf817fa700 10 client.25153 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:56.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7faf817fa700 1 -- 172.21.15.99:0/34869193 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55766210c4f0 con 0x55766215e2f0 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7faf817fa700 10 client.25153 handle_command_reply: tid=0 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 client.25153 shutdown 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55766214a2a0 msgr2=0x55766214dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55766214a2a0 0x55766214dc90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf880dff70 tx=0x7faf880aa060 comp rx=0 tx=0).stop 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 shutdown_connections 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55766215e2f0 0x5576621606e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x55766214e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557662144f70 0x5576621508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 --2- 172.21.15.99:0/34869193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55766214a2a0 0x55766214dc90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 >> 172.21.15.99:0/34869193 conn(0x55766206b3e0 msgr2=0x557662135860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7faf817fa700 0 client.25153 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 shutdown_connections 2024-05-14T17:05:56.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 1 -- 172.21.15.99:0/34869193 wait complete. 2024-05-14T17:05:56.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.158+0000 7fafaa944b80 20 client.25153 trim_cache size 0 max 16384 2024-05-14T17:05:56.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.178+0000 7fafaa944b80 1 Processor -- start 2024-05-14T17:05:56.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.178+0000 7fafaa944b80 1 -- start start 2024-05-14T17:05:56.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x55766215a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662135970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7fafaa944b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf981ae1d0 con 0x55766214abc0 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7fafaa944b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf981aba60 con 0x55766214a2a0 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7fafaa944b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf981a98d0 con 0x557662144f70 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9e64b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x55766215a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9e64b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662165f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43350/0 (socket says 172.21.15.99:43350) 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9e64b700 1 -- 172.21.15.99:0/596600735 learned_addr learned my addr 172.21.15.99:0/596600735 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x55766215a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:34032/0 (socket says 172.21.15.99:34032) 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9d649700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662135970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 -- 172.21.15.99:0/596600735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 msgr2=0x557662135970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662135970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 -- 172.21.15.99:0/596600735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 msgr2=0x557662165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9de4a700 1 -- 172.21.15.99:0/596600735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557662133500 con 0x557662144f70 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9e64b700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:56.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.179+0000 7faf9d649700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662135970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:56.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7faf9de4a700 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x55766215a470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911089d66552fbd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7faf837fe700 1 -- 172.21.15.99:0/596600735 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf940043d0 con 0x557662144f70 2024-05-14T17:05:56.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7faf837fe700 1 -- 172.21.15.99:0/596600735 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9400fb70 con 0x557662144f70 2024-05-14T17:05:56.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7faf837fe700 1 -- 172.21.15.99:0/596600735 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94003da0 con 0x557662144f70 2024-05-14T17:05:56.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 -- 172.21.15.99:0/596600735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 msgr2=0x55766215a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x55766215a470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 -- 172.21.15.99:0/596600735 shutdown_connections 2024-05-14T17:05:56.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x55766215a470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 --2- 172.21.15.99:0/596600735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662135970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.180+0000 7fafaa944b80 1 -- 172.21.15.99:0/596600735 >> 172.21.15.99:0/596600735 conn(0x55766206b3e0 msgr2=0x5576621495d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 -- 172.21.15.99:0/596600735 shutdown_connections 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 -- 172.21.15.99:0/596600735 wait complete. 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 -- start start 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 Processor -- start 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662153d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf981a9f20 con 0x55766214abc0 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf981ab390 con 0x55766214a2a0 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf981aa5f0 con 0x557662144f70 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.181+0000 7fafaa944b80 10 client.0 _ll_register_callbacks cb 0x557662025710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:56.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:34048/0 (socket says 172.21.15.99:34048) 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9d649700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 -- 172.21.15.99:0/3894337106 learned_addr learned my addr 172.21.15.99:0/3894337106 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9e64b700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 -- 172.21.15.99:0/3894337106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 msgr2=0x557662153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 -- 172.21.15.99:0/3894337106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 msgr2=0x557662154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 -- 172.21.15.99:0/3894337106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576621c8b30 con 0x557662144f70 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9d649700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf9de4a700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621537c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b7f0 tx=0x7faf94009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1be1fcdab3f2401 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf817fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf817fa700 1 -- 172.21.15.99:0/3894337106 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9404a060 con 0x557662144f70 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf817fa700 1 -- 172.21.15.99:0/3894337106 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7faf9400f640 con 0x557662144f70 2024-05-14T17:05:56.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9400fb70 con 0x557662144f70 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7faf817fa700 1 -- 172.21.15.99:0/3894337106 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf9400f840 con 0x557662144f70 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7fafaa944b80 10 client.25159 fetch_fsmap learned FSMap version 26 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.182+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576621c9d70 con 0x557662144f70 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7faf817fa700 1 -- 172.21.15.99:0/3894337106 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9400fb70 con 0x557662144f70 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7faf817fa700 1 -- 172.21.15.99:0/3894337106 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faf9403d2a0 con 0x557662144f70 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7fafaa944b80 10 client.25159 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7fafaa944b80 10 client.25159 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7fafaa944b80 20 client.25159 populate_metadata read hostname 'smithi099' 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7fafaa944b80 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5576621c9ff0 0x5576621cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7fafaa944b80 4 client.25159 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}] 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}) -- 0x5576620a0d70 con 0x5576621c9ff0 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7faf9d649700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5576621c9ff0 0x5576621cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:56.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7faf9d649700 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5576621c9ff0 0x5576621cc3e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64698f448ad5268b server_cookie=1fce0a7aa0ec5ad5 in_seq=0 out_seq=0 2024-05-14T17:05:56.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.183+0000 7faf817fa700 10 client.25159 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:56.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7faf817fa700 1 -- 172.21.15.99:0/3894337106 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5576620a0d70 con 0x5576621c9ff0 2024-05-14T17:05:56.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7faf817fa700 10 client.25159 handle_command_reply: tid=0 2024-05-14T17:05:56.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 client.25159 shutdown 2024-05-14T17:05:56.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 msgr2=0x5576621537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621537c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b7f0 tx=0x7faf94009310 comp rx=0 tx=0).stop 2024-05-14T17:05:56.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 shutdown_connections 2024-05-14T17:05:56.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5576621c9ff0 0x5576621cc3e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55766214abc0 0x557662154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557662144f70 0x5576621537c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7faf817fa700 0 client.25159 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:05:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 --2- 172.21.15.99:0/3894337106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55766214a2a0 0x557662153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 >> 172.21.15.99:0/3894337106 conn(0x55766206b3e0 msgr2=0x55766214c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 shutdown_connections 2024-05-14T17:05:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 1 -- 172.21.15.99:0/3894337106 wait complete. 2024-05-14T17:05:56.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.338+0000 7fafaa944b80 20 client.25159 trim_cache size 0 max 16384 2024-05-14T17:05:56.341 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 6, 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "b889149b": { 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 6, 2024-05-14T17:05:56.342 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 146.7, 2024-05-14T17:05:56.343 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faf840b2b20 msgr2=0x7faf840b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faf840b2b20 0x7faf840b4fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf94006010 tx=0x7faf9401a040 comp rx=0 tx=0).stop 2024-05-14T17:05:56.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 msgr2=0x7faf9819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 0x7faf9819fc90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faf8800e9c0 tx=0x7faf8800ecd0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 shutdown_connections 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faf840b2b20 0x7faf840b4fe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf98108620 0x7faf98199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf98101f00 0x7faf9819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 --2- 172.21.15.99:0/4045878827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf981028c0 0x7faf9819fc90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 >> 172.21.15.99:0/4045878827 conn(0x7faf98077780 msgr2=0x7faf981008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 shutdown_connections 2024-05-14T17:05:56.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:05:56.339+0000 7fafa00ae700 1 -- 172.21.15.99:0/4045878827 wait complete. 2024-05-14T17:05:56.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 16.97 MB/sec warmup 8 sec latency 294.337 ms 2024-05-14T17:05:56.995 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b889149b', quiesce took 5.6 seconds. Will release after: 21.1 2024-05-14T17:05:57.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:56 smithi150 ceph-mon[36204]: pgmap v72: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 729 KiB/s wr, 134 op/s 2024-05-14T17:05:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:56 smithi099 ceph-mon[31851]: pgmap v72: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 729 KiB/s wr, 134 op/s 2024-05-14T17:05:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:56 smithi193 ceph-mon[35962]: pgmap v72: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 729 KiB/s wr, 134 op/s 2024-05-14T17:05:57.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 15.08 MB/sec warmup 9 sec latency 1294.462 ms 2024-05-14T17:05:58.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 13.57 MB/sec warmup 10 sec latency 2294.518 ms 2024-05-14T17:05:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:05:58 smithi099 ceph-mon[31851]: pgmap v73: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 729 KiB/s wr, 134 op/s 2024-05-14T17:05:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:05:58 smithi193 ceph-mon[35962]: pgmap v73: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 729 KiB/s wr, 134 op/s 2024-05-14T17:05:59.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:05:58 smithi150 ceph-mon[36204]: pgmap v73: 97 pgs: 97 active+clean; 8.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 729 KiB/s wr, 134 op/s 2024-05-14T17:05:59.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 12.34 MB/sec warmup 11 sec latency 3294.583 ms 2024-05-14T17:06:00.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 11.31 MB/sec warmup 12 sec latency 4294.663 ms 2024-05-14T17:06:01.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:00 smithi099 ceph-mon[31851]: pgmap v74: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-05-14T17:06:01.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:00 smithi193 ceph-mon[35962]: pgmap v74: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-05-14T17:06:01.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:00 smithi150 ceph-mon[36204]: pgmap v74: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-05-14T17:06:01.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 10.44 MB/sec warmup 13 sec latency 5294.760 ms 2024-05-14T17:06:02.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 9.70 MB/sec warmup 14 sec latency 6294.876 ms 2024-05-14T17:06:03.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:02 smithi099 ceph-mon[31851]: pgmap v75: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-14T17:06:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:02 smithi193 ceph-mon[35962]: pgmap v75: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-14T17:06:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:02 smithi150 ceph-mon[36204]: pgmap v75: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-05-14T17:06:03.516 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 9.05 MB/sec warmup 15 sec latency 7294.982 ms 2024-05-14T17:06:04.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 8.48 MB/sec warmup 16 sec latency 8295.065 ms 2024-05-14T17:06:05.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:04 smithi099 ceph-mon[31851]: pgmap v76: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-05-14T17:06:05.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:04 smithi193 ceph-mon[35962]: pgmap v76: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-05-14T17:06:05.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:04 smithi150 ceph-mon[36204]: pgmap v76: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-05-14T17:06:05.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 7.98 MB/sec warmup 17 sec latency 9295.160 ms 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 -- 172.21.15.99:0/3841046745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c06bc60 msgr2=0x7f930c10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 --2- 172.21.15.99:0/3841046745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c06bc60 0x7f930c10f890 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9304009ca0 tx=0x7f930402a8c0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 -- 172.21.15.99:0/3841046745 shutdown_connections 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 --2- 172.21.15.99:0/3841046745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c06bc60 0x7f930c10f890 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 --2- 172.21.15.99:0/3841046745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06b2c0 0x7f930c06b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 --2- 172.21.15.99:0/3841046745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c108a40 0x7f930c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 -- 172.21.15.99:0/3841046745 >> 172.21.15.99:0/3841046745 conn(0x7f930c077750 msgr2=0x7f930c077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 -- 172.21.15.99:0/3841046745 shutdown_connections 2024-05-14T17:06:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.629+0000 7f931221c640 1 -- 172.21.15.99:0/3841046745 wait complete. 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 Processor -- start 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 -- start start 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c06b2c0 0x7f930c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06bc60 0x7f930c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 0x7f930c1027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f930c112ce0 con 0x7f930c108a40 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f930c112b60 con 0x7f930c06bc60 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f931221c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f930c112e60 con 0x7f930c06b2c0 2024-05-14T17:06:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f930bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 0x7f930c1027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f930bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 0x7f930c1027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60916/0 (socket says 172.21.15.99:60916) 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f930bfff640 1 -- 172.21.15.99:0/3792145624 learned_addr learned my addr 172.21.15.99:0/3792145624 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f9303fff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06bc60 0x7f930c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.630+0000 7f930b7fe640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c06b2c0 0x7f930c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f930bfff640 1 -- 172.21.15.99:0/3792145624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c06b2c0 msgr2=0x7f930c101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f930bfff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c06b2c0 0x7f930c101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f930bfff640 1 -- 172.21.15.99:0/3792145624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06bc60 msgr2=0x7f930c1022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f930bfff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06bc60 0x7f930c1022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f930bfff640 1 -- 172.21.15.99:0/3792145624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1a75a0 con 0x7f930c108a40 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f9303fff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06bc60 0x7f930c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f930bfff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 0x7f930c1027f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f93040098d0 tx=0x7f93040317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f93097fa640 1 -- 172.21.15.99:0/3792145624 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304003b70 con 0x7f930c108a40 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f93097fa640 1 -- 172.21.15.99:0/3792145624 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9304003d10 con 0x7f930c108a40 2024-05-14T17:06:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c1a7830 con 0x7f930c108a40 2024-05-14T17:06:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f93097fa640 1 -- 172.21.15.99:0/3792145624 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930402fa20 con 0x7f930c108a40 2024-05-14T17:06:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.631+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1a7dd0 con 0x7f930c108a40 2024-05-14T17:06:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.633+0000 7f93097fa640 1 -- 172.21.15.99:0/3792145624 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f930402fc00 con 0x7f930c108a40 2024-05-14T17:06:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 Processor -- start 2024-05-14T17:06:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 -- start start 2024-05-14T17:06:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c12558cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12563590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12565ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560c124cbc30 con 0x560c125588f0 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560c124bb7d0 con 0x560c12563ad0 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f931501c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c124bf810 con 0x560c12559210 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c12558cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c12558cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60920/0 (socket says 172.21.15.99:60920) 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 -- 172.21.15.99:0/3563470080 learned_addr learned my addr 172.21.15.99:0/3563470080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f930b7fe640 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12563590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f93097fa640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92f80b2ac0 0x7f92f80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 -- 172.21.15.99:0/3563470080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 msgr2=0x560c12563590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12563590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 -- 172.21.15.99:0/3563470080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 msgr2=0x560c12565ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12565ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.634+0000 7f9303fff640 1 -- 172.21.15.99:0/3563470080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c121d0210 con 0x560c125588f0 2024-05-14T17:06:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.635+0000 7f930bfff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92f80b2ac0 0x7f92f80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.635+0000 7f9303fff640 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c12558cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92f4008cc0 tx=0x7f92f4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe1a03f443ca55c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.635+0000 7f93097fa640 1 -- 172.21.15.99:0/3792145624 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f9304103930 con 0x7f930c108a40 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.635+0000 7f9301ffb640 1 -- 172.21.15.99:0/3563470080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4016070 con 0x560c125588f0 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.635+0000 7f9301ffb640 1 -- 172.21.15.99:0/3563470080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f40056d0 con 0x560c125588f0 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.635+0000 7f9301ffb640 1 -- 172.21.15.99:0/3563470080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f40126e0 con 0x560c125588f0 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 -- 172.21.15.99:0/3563470080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 msgr2=0x560c12558cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f930bfff640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92f80b2ac0 0x7f92f80b4f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9304101270 tx=0x7f930402a8c0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c12558cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92f4008cc0 tx=0x7f92f4010740 comp rx=0 tx=0).stop 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 -- 172.21.15.99:0/3563470080 shutdown_connections 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12565ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12563590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 --2- 172.21.15.99:0/3563470080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c12558cd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 -- 172.21.15.99:0/3563470080 >> 172.21.15.99:0/3563470080 conn(0x560c12554e00 msgr2=0x560c125551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 -- 172.21.15.99:0/3563470080 shutdown_connections 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 -- 172.21.15.99:0/3563470080 wait complete. 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 -- start start 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.636+0000 7f931501c740 1 Processor -- start 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c125633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 0x560c125606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12560c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560c121a1790 con 0x560c12559210 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560c12477890 con 0x560c12563ad0 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c125667a0 con 0x560c125588f0 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 10 client.0 _ll_register_callbacks cb 0x560c1247ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 0x560c125606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 0x560c125606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60926/0 (socket says 172.21.15.99:60926) 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 -- 172.21.15.99:0/1329512043 learned_addr learned my addr 172.21.15.99:0/1329512043 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f9303fff640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c125633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f9303fff640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12560c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 -- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 msgr2=0x560c125633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c125633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 -- 172.21.15.99:0/1329512043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 msgr2=0x560c12560c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12560c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 -- 172.21.15.99:0/1329512043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c1229c620 con 0x560c12559210 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f9303fff640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12560c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f930b7fe640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 0x560c125606d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b920 tx=0x7f92fc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b80eb602bf1e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f92eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f92eaffd640 1 -- 172.21.15.99:0/1329512043 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc003ea0 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f92eaffd640 1 -- 172.21.15.99:0/1329512043 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f92fc01b640 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.637+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f40056d0 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f92eaffd640 1 -- 172.21.15.99:0/1329512043 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f92fc01b840 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 10 client.15585 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c125721d0 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f92eaffd640 1 -- 172.21.15.99:0/1329512043 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f40056d0 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f92eaffd640 1 -- 172.21.15.99:0/1329512043 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f92fc019020 con 0x560c12559210 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 20 client.15585 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c12573690 0x560c12575a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 4 client.15585 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c124cdb80 con 0x560c12573690 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.638+0000 7f9303fff640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c12573690 0x560c12575a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.639+0000 7f92eaffd640 10 client.15585 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.639+0000 7f9303fff640 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c12573690 0x560c12575a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7400cd4b714ab98f server_cookie=8b02880d7a5c9689 in_seq=0 out_seq=0 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.639+0000 7f92eaffd640 1 -- 172.21.15.99:0/1329512043 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c124cdb80 con 0x560c12573690 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.639+0000 7f92eaffd640 10 client.15585 handle_command_reply: tid=0 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.639+0000 7f931501c740 1 client.15585 shutdown 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 msgr2=0x560c125606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 0x560c125606d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b920 tx=0x7f92fc00bde0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 shutdown_connections 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c12573690 0x560c12575a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c12560c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12559210 0x560c125606d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 --2- 172.21.15.99:0/1329512043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c125633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f92eaffd640 0 client.15585 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 >> 172.21.15.99:0/1329512043 conn(0x560c12554e00 msgr2=0x560c12554100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 shutdown_connections 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 1 -- 172.21.15.99:0/1329512043 wait complete. 2024-05-14T17:06:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.640+0000 7f931501c740 20 client.15585 trim_cache size 0 max 16384 2024-05-14T17:06:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 Processor -- start 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 -- start start 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c1256f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12562f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c1257b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1b2f30 con 0x560c125588f0 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1b0800 con 0x560c12563ad0 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f931501c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1ae650 con 0x560c12559210 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f9303fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c1256f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f9303fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c1256f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60938/0 (socket says 172.21.15.99:60938) 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f9303fff640 1 -- 172.21.15.99:0/1745354401 learned_addr learned my addr 172.21.15.99:0/1745354401 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.655+0000 7f9303fff640 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c1257b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f930b7fe640 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12562f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f9303fff640 1 -- 172.21.15.99:0/1745354401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 msgr2=0x560c12562f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f9303fff640 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12562f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f9303fff640 1 -- 172.21.15.99:0/1745354401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 msgr2=0x560c1257b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f9303fff640 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c1257b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f9303fff640 1 -- 172.21.15.99:0/1745354401 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c12560f70 con 0x560c125588f0 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f9303fff640 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c1256f670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f92f4010c40 tx=0x7f92f4008310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9898dcc2555c02f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f93017fa640 1 -- 172.21.15.99:0/1745354401 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4016040 con 0x560c125588f0 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f93017fa640 1 -- 172.21.15.99:0/1745354401 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f4004680 con 0x560c125588f0 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.656+0000 7f93017fa640 1 -- 172.21.15.99:0/1745354401 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f402a5c0 con 0x560c125588f0 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- 172.21.15.99:0/1745354401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 msgr2=0x560c1256f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c1256f670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f92f4010c40 tx=0x7f92f4008310 comp rx=0 tx=0).stop 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- 172.21.15.99:0/1745354401 shutdown_connections 2024-05-14T17:06:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12563ad0 0x560c1257b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c12559210 0x560c12562f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- 172.21.15.99:0/1745354401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c125588f0 0x560c1256f670 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- 172.21.15.99:0/1745354401 >> 172.21.15.99:0/1745354401 conn(0x560c12554e00 msgr2=0x560c1255ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- 172.21.15.99:0/1745354401 shutdown_connections 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- 172.21.15.99:0/1745354401 wait complete. 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- start start 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 Processor -- start 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c1255d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12559210 0x560c1255d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 0x560c1255def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1aecc0 con 0x560c12563ad0 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1b0130 con 0x560c12559210 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1af390 con 0x560c125588f0 2024-05-14T17:06:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.657+0000 7f931501c740 10 client.0 _ll_register_callbacks cb 0x7f92fc004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 0x560c1255def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 0x560c1255def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60948/0 (socket says 172.21.15.99:60948) 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 -- 172.21.15.99:0/382347424 learned_addr learned my addr 172.21.15.99:0/382347424 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c1255d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f930b7fe640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12559210 0x560c1255d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 -- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 msgr2=0x560c1255d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c1255d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 -- 172.21.15.99:0/382347424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12559210 msgr2=0x560c1255d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12559210 0x560c1255d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 -- 172.21.15.99:0/382347424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c125dcf00 con 0x560c12563ad0 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f9303fff640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 0x560c1255def0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f92f4010fd0 tx=0x7f92f402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc53cd829266e35c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f92eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f92eaffd640 1 -- 172.21.15.99:0/382347424 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4004a40 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f92eaffd640 1 -- 172.21.15.99:0/382347424 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f92f4004be0 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f4004680 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f92eaffd640 1 -- 172.21.15.99:0/382347424 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f92f402a360 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f931501c740 10 client.15591 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c125de150 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.658+0000 7f92eaffd640 1 -- 172.21.15.99:0/382347424 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f4004680 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f92eaffd640 1 -- 172.21.15.99:0/382347424 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f92f402a040 con 0x560c12563ad0 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f931501c740 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f931501c740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f931501c740 20 client.15591 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f931501c740 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c125de650 0x560c125e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f931501c740 4 client.15591 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c124c7360 con 0x560c125de650 2024-05-14T17:06:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f930b7fe640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c125de650 0x560c125e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f930b7fe640 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c125de650 0x560c125e0a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d495c0d8151f148 server_cookie=55c121c99e1f3f4e in_seq=0 out_seq=0 2024-05-14T17:06:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.659+0000 7f92eaffd640 10 client.15591 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f92eaffd640 1 -- 172.21.15.99:0/382347424 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560c124c7360 con 0x560c125de650 2024-05-14T17:06:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f92eaffd640 10 client.15591 handle_command_reply: tid=0 2024-05-14T17:06:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 client.15591 shutdown 2024-05-14T17:06:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 msgr2=0x560c1255def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 0x560c1255def0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f92f4010fd0 tx=0x7f92f402e040 comp rx=0 tx=0).stop 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 shutdown_connections 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c125de650 0x560c125e0a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c12563ad0 0x560c1255def0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c12559210 0x560c1255d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f92eaffd640 0 client.15591 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 --2- 172.21.15.99:0/382347424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c125588f0 0x560c1255d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 >> 172.21.15.99:0/382347424 conn(0x560c12554e00 msgr2=0x560c1255a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 shutdown_connections 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 1 -- 172.21.15.99:0/382347424 wait complete. 2024-05-14T17:06:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.708+0000 7f931501c740 20 client.15591 trim_cache size 0 max 16384 2024-05-14T17:06:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:06:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:06:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92f80b2ac0 msgr2=0x7f92f80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92f80b2ac0 0x7f92f80b4f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9304101270 tx=0x7f930402a8c0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 msgr2=0x7f930c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 0x7f930c1027f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f93040098d0 tx=0x7f93040317b0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 shutdown_connections 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92f80b2ac0 0x7f92f80b4f80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f930c108a40 0x7f930c1027f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f930c06bc60 0x7f930c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 --2- 172.21.15.99:0/3792145624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f930c06b2c0 0x7f930c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 >> 172.21.15.99:0/3792145624 conn(0x7f930c077750 msgr2=0x7f930c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 shutdown_connections 2024-05-14T17:06:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:05.709+0000 7f931221c640 1 -- 172.21.15.99:0/3792145624 wait complete. 2024-05-14T17:06:06.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 7.54 MB/sec warmup 18 sec latency 10295.263 ms 2024-05-14T17:06:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:06 smithi099 ceph-mon[31851]: pgmap v77: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 843 KiB/s wr, 71 op/s 2024-05-14T17:06:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:06 smithi193 ceph-mon[35962]: pgmap v77: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 843 KiB/s wr, 71 op/s 2024-05-14T17:06:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:06 smithi150 ceph-mon[36204]: pgmap v77: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 843 KiB/s wr, 71 op/s 2024-05-14T17:06:07.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 7.14 MB/sec warmup 19 sec latency 11295.319 ms 2024-05-14T17:06:08.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 6.79 MB/sec warmup 20 sec latency 12295.418 ms 2024-05-14T17:06:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:08 smithi099 ceph-mon[31851]: pgmap v78: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 843 KiB/s wr, 71 op/s 2024-05-14T17:06:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:08 smithi193 ceph-mon[35962]: pgmap v78: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 843 KiB/s wr, 71 op/s 2024-05-14T17:06:09.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:08 smithi150 ceph-mon[36204]: pgmap v78: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 843 KiB/s wr, 71 op/s 2024-05-14T17:06:09.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 6.46 MB/sec warmup 21 sec latency 13295.514 ms 2024-05-14T17:06:10.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 6.17 MB/sec warmup 22 sec latency 14295.549 ms 2024-05-14T17:06:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:10 smithi099 ceph-mon[31851]: pgmap v79: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 863 KiB/s wr, 110 op/s 2024-05-14T17:06:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:10 smithi193 ceph-mon[35962]: pgmap v79: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 863 KiB/s wr, 110 op/s 2024-05-14T17:06:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:10 smithi150 ceph-mon[36204]: pgmap v79: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 863 KiB/s wr, 110 op/s 2024-05-14T17:06:11.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 5.90 MB/sec warmup 23 sec latency 15295.671 ms 2024-05-14T17:06:12.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 5.66 MB/sec warmup 24 sec latency 16295.747 ms 2024-05-14T17:06:13.255 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:12 smithi150 ceph-mon[36204]: pgmap v80: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 KiB/s wr, 58 op/s 2024-05-14T17:06:13.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:12 smithi099 ceph-mon[31851]: pgmap v80: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 KiB/s wr, 58 op/s 2024-05-14T17:06:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:12 smithi193 ceph-mon[35962]: pgmap v80: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 KiB/s wr, 58 op/s 2024-05-14T17:06:13.517 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 5.43 MB/sec warmup 25 sec latency 17295.853 ms 2024-05-14T17:06:14.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 5.22 MB/sec warmup 26 sec latency 18295.967 ms 2024-05-14T17:06:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:14 smithi099 ceph-mon[31851]: pgmap v81: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 50 KiB/s wr, 61 op/s 2024-05-14T17:06:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:14 smithi193 ceph-mon[35962]: pgmap v81: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 50 KiB/s wr, 61 op/s 2024-05-14T17:06:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:14 smithi150 ceph-mon[36204]: pgmap v81: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 50 KiB/s wr, 61 op/s 2024-05-14T17:06:15.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 5.03 MB/sec warmup 27 sec latency 19296.062 ms 2024-05-14T17:06:16.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 4.85 MB/sec warmup 28 sec latency 20296.147 ms 2024-05-14T17:06:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:16 smithi099 ceph-mon[31851]: pgmap v82: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 20 KiB/s wr, 41 op/s 2024-05-14T17:06:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:16 smithi193 ceph-mon[35962]: pgmap v82: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 20 KiB/s wr, 41 op/s 2024-05-14T17:06:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:16 smithi150 ceph-mon[36204]: pgmap v82: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 20 KiB/s wr, 41 op/s 2024-05-14T17:06:17.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 4.68 MB/sec warmup 29 sec latency 21296.215 ms 2024-05-14T17:06:18.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:06:18.345 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:18.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 4.52 MB/sec warmup 30 sec latency 22296.291 ms 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 -- 172.21.15.99:0/2293798844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab8101ca0 msgr2=0x7f4ab8102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 --2- 172.21.15.99:0/2293798844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab8101ca0 0x7f4ab8102120 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009900 tx=0x7f4aa802a410 comp rx=0 tx=0).stop 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 -- 172.21.15.99:0/2293798844 shutdown_connections 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 --2- 172.21.15.99:0/2293798844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab8102660 0x7f4ab810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 --2- 172.21.15.99:0/2293798844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab8101ca0 0x7f4ab8102120 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 --2- 172.21.15.99:0/2293798844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ab81068f0 0x7f4ab8106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 -- 172.21.15.99:0/2293798844 >> 172.21.15.99:0/2293798844 conn(0x7f4ab8077750 msgr2=0x7f4ab8077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 -- 172.21.15.99:0/2293798844 shutdown_connections 2024-05-14T17:06:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.799+0000 7f4ac047e640 1 -- 172.21.15.99:0/2293798844 wait complete. 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.800+0000 7f4ac047e640 1 Processor -- start 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.800+0000 7f4ac047e640 1 -- start start 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.800+0000 7f4ac047e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ab8101ca0 0x7f4ab8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab8102660 0x7f4ab819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 0x7f4ab81a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab8111bd0 con 0x7f4ab81068f0 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab8111a50 con 0x7f4ab8102660 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab8111d50 con 0x7f4ab8101ca0 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 0x7f4ab81a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 0x7f4ab81a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43168/0 (socket says 172.21.15.99:43168) 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 -- 172.21.15.99:0/3947860900 learned_addr learned my addr 172.21.15.99:0/3947860900 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ab8101ca0 0x7f4ab8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abda19640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab8102660 0x7f4ab819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 -- 172.21.15.99:0/3947860900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ab8101ca0 msgr2=0x7f4ab8199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ab8101ca0 0x7f4ab8199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 -- 172.21.15.99:0/3947860900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab8102660 msgr2=0x7f4ab819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab8102660 0x7f4ab819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 -- 172.21.15.99:0/3947860900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab81a5510 con 0x7f4ab81068f0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4abea1b640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 0x7f4ab81a4fd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009fd0 tx=0x7f4ab000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4aaf7fe640 1 -- 172.21.15.99:0/3947860900 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0017070 con 0x7f4ab81068f0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab81a56e0 con 0x7f4ab81068f0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4aaf7fe640 1 -- 172.21.15.99:0/3947860900 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab0003e20 con 0x7f4ab81068f0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4aaf7fe640 1 -- 172.21.15.99:0/3947860900 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0010460 con 0x7f4ab81068f0 2024-05-14T17:06:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.801+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab81a5bb0 con 0x7f4ab81068f0 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4aaf7fe640 1 -- 172.21.15.99:0/3947860900 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4ab0003970 con 0x7f4ab81068f0 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 Processor -- start 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 -- start start 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7a8f0 0x5621ddd7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7b210 0x5621ddd85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621ddcedc30 con 0x5621ddd85ad0 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5621ddcdd7d0 con 0x5621ddd7a8f0 2024-05-14T17:06:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.804+0000 7f4ac319b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621ddce1810 con 0x5621ddd7b210 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd87ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43176/0 (socket says 172.21.15.99:43176) 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 -- 172.21.15.99:0/3777194583 learned_addr learned my addr 172.21.15.99:0/3777194583 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7b210 0x5621ddd85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7a8f0 0x5621ddd7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 -- 172.21.15.99:0/3777194583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7b210 msgr2=0x5621ddd85590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7b210 0x5621ddd85590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 -- 172.21.15.99:0/3777194583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7a8f0 msgr2=0x5621ddd7acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7a8f0 0x5621ddd7acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 -- 172.21.15.99:0/3777194583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621dd9f2210 con 0x5621ddd85ad0 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abda19640 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd87ec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4aa802a930 tx=0x7f4aa8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8923d09af848d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4aaf7fe640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4aa40b2a10 0x7f4aa40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4aa40b2a10 0x7f4aa40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4aacbfe640 1 -- 172.21.15.99:0/3777194583 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8035070 con 0x5621ddd85ad0 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4aacbfe640 1 -- 172.21.15.99:0/3777194583 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa8045070 con 0x5621ddd85ad0 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4aacbfe640 1 -- 172.21.15.99:0/3777194583 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8003b20 con 0x5621ddd85ad0 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.805+0000 7f4aaf7fe640 1 -- 172.21.15.99:0/3947860900 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f4ab00e2300 con 0x7f4ab81068f0 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 -- 172.21.15.99:0/3777194583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 msgr2=0x5621ddd87ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4aa40b2a10 0x7f4aa40b4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5621ddd7be90 tx=0x7f4ab400b390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd87ec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4aa802a930 tx=0x7f4aa8005b50 comp rx=0 tx=0).stop 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 -- 172.21.15.99:0/3777194583 shutdown_connections 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd87ec0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7b210 0x5621ddd85590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3777194583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7a8f0 0x5621ddd7acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 -- 172.21.15.99:0/3777194583 >> 172.21.15.99:0/3777194583 conn(0x5621ddd76e00 msgr2=0x5621ddd771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 -- 172.21.15.99:0/3777194583 shutdown_connections 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.806+0000 7f4ac319b740 1 -- 172.21.15.99:0/3777194583 wait complete. 2024-05-14T17:06:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 -- start start 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 Processor -- start 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd82db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621dd9c3790 con 0x5621ddd7b210 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5621ddc99890 con 0x5621ddd85ad0 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621ddd88740 con 0x5621ddd7a8f0 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4ac319b740 10 client.0 _ll_register_callbacks cb 0x5621ddc9cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd827a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43190/0 (socket says 172.21.15.99:43190) 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 -- 172.21.15.99:0/3895074301 learned_addr learned my addr 172.21.15.99:0/3895074301 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abea1b640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd82db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 -- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 msgr2=0x5621ddd854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 -- 172.21.15.99:0/3895074301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 msgr2=0x5621ddd82db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd82db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.807+0000 7f4abda19640 1 -- 172.21.15.99:0/3895074301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ddabe620 con 0x5621ddd7b210 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4abda19640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd827a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005ec0 tx=0x7f4aa8003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fb7e9cea7e5241 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/3895074301 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa804d070 con 0x5621ddd7b210 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/3895074301 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f4aa8005450 con 0x5621ddd7b210 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa8045070 con 0x5621ddd7b210 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/3895074301 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4aa8035040 con 0x5621ddd7b210 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 10 client.15603 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621ddd941e0 con 0x5621ddd7b210 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/3895074301 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa8045070 con 0x5621ddd7b210 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/3895074301 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4aa8052460 con 0x5621ddd7b210 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 20 client.15603 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621ddd95530 0x5621ddd97920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 4 client.15603 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.808+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621ddcefb80 con 0x5621ddd95530 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.809+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621ddd95530 0x5621ddd97920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.809+0000 7f4aa1ffb640 10 client.15603 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.809+0000 7f4abe21a640 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621ddd95530 0x5621ddd97920 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baca954ba8f7d8c3 server_cookie=edf0aaaeeeee756 in_seq=0 out_seq=0 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/3895074301 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5621ddcefb80 con 0x5621ddd95530 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4aa1ffb640 10 client.15603 handle_command_reply: tid=0 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 client.15603 shutdown 2024-05-14T17:06:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 msgr2=0x5621ddd827a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd827a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005ec0 tx=0x7f4aa8003700 comp rx=0 tx=0).stop 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 shutdown_connections 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621ddd95530 0x5621ddd97920 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd82db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd827a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4aa1ffb640 0 client.15603 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 --2- 172.21.15.99:0/3895074301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 >> 172.21.15.99:0/3895074301 conn(0x5621ddd76e00 msgr2=0x5621ddd76100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 shutdown_connections 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 1 -- 172.21.15.99:0/3895074301 wait complete. 2024-05-14T17:06:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.810+0000 7f4ac319b740 20 client.15603 trim_cache size 0 max 16384 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 Processor -- start 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 -- start start 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd81900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd91610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81b0ca0 con 0x5621ddd7b210 2024-05-14T17:06:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81ae570 con 0x5621ddd85ad0 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4ac319b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81ac3c0 con 0x5621ddd7a8f0 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4abda19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd91610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4abda19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd91610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43196/0 (socket says 172.21.15.99:43196) 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.825+0000 7f4abda19640 1 -- 172.21.15.99:0/1866249820 learned_addr learned my addr 172.21.15.99:0/1866249820 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abe21a640 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd81900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abda19640 1 -- 172.21.15.99:0/1866249820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 msgr2=0x5621ddd81900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abda19640 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd81900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abda19640 1 -- 172.21.15.99:0/1866249820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 msgr2=0x5621ddd91b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abda19640 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abda19640 1 -- 172.21.15.99:0/1866249820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ddd7b040 con 0x5621ddd7b210 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4abda19640 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd91610 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8052880 tx=0x7f4aa8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8419c72162f749 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4aa3fff640 1 -- 172.21.15.99:0/1866249820 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa805f020 con 0x5621ddd7b210 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4aa3fff640 1 -- 172.21.15.99:0/1866249820 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa8045070 con 0x5621ddd7b210 2024-05-14T17:06:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.826+0000 7f4aa3fff640 1 -- 172.21.15.99:0/1866249820 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa804d040 con 0x5621ddd7b210 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- 172.21.15.99:0/1866249820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 msgr2=0x5621ddd91610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd91610 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8052880 tx=0x7f4aa8005fd0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- 172.21.15.99:0/1866249820 shutdown_connections 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd85ad0 0x5621ddd91b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd7b210 0x5621ddd91610 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- 172.21.15.99:0/1866249820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd81900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- 172.21.15.99:0/1866249820 >> 172.21.15.99:0/1866249820 conn(0x5621ddd76e00 msgr2=0x5621ddd83fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- 172.21.15.99:0/1866249820 shutdown_connections 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- 172.21.15.99:0/1866249820 wait complete. 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- start start 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 Processor -- start 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7b210 0x5621ddd8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd8b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81aca30 con 0x5621ddd85ad0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81adea0 con 0x5621ddd7b210 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81ad100 con 0x5621ddd7a8f0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4ac319b740 10 client.0 _ll_register_callbacks cb 0x7f4aa8009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4abea1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd8b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4abea1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd8b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43206/0 (socket says 172.21.15.99:43206) 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.827+0000 7f4abea1b640 1 -- 172.21.15.99:0/2632523375 learned_addr learned my addr 172.21.15.99:0/2632523375 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abda19640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7b210 0x5621ddd8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abe21a640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd8dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abea1b640 1 -- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 msgr2=0x5621ddd8dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abea1b640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd8dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abea1b640 1 -- 172.21.15.99:0/2632523375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7b210 msgr2=0x5621ddd8afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abea1b640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7b210 0x5621ddd8afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abea1b640 1 -- 172.21.15.99:0/2632523375 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ddd83120 con 0x5621ddd85ad0 2024-05-14T17:06:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4abea1b640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd8b4e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00a93c0 tx=0x7f4ab00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571ad28f6781abed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/2632523375 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0020070 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/2632523375 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f4ab00a5cb0 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa8045070 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/2632523375 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ab00a96c0 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4ac319b740 10 client.15609 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621dddff740 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.828+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/2632523375 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa8045070 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/2632523375 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4ab00b1fe0 con 0x5621ddd85ad0 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4ac319b740 10 client.15609 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4ac319b740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4ac319b740 20 client.15609 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4ac319b740 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621dddfff00 0x5621dde02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4ac319b740 4 client.15609 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621ddce9360 con 0x5621dddfff00 2024-05-14T17:06:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4abe21a640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621dddfff00 0x5621dde02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4abe21a640 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621dddfff00 0x5621dde02320 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3117e8b16f8e0f server_cookie=f210644edc153645 in_seq=0 out_seq=0 2024-05-14T17:06:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.829+0000 7f4aa1ffb640 10 client.15609 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.871+0000 7f4aa1ffb640 1 -- 172.21.15.99:0/2632523375 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5621ddce9360 con 0x5621dddfff00 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.871+0000 7f4aa1ffb640 10 client.15609 handle_command_reply: tid=0 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.871+0000 7f4ac319b740 1 client.15609 shutdown 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 msgr2=0x5621ddd8b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd8b4e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00a93c0 tx=0x7f4ab00042e0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 shutdown_connections 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5621dddfff00 0x5621dde02320 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ddd85ad0 0x5621ddd8b4e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621ddd7b210 0x5621ddd8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4aa1ffb640 0 client.15609 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 --2- 172.21.15.99:0/2632523375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621ddd7a8f0 0x5621ddd8dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 >> 172.21.15.99:0/2632523375 conn(0x5621ddd76e00 msgr2=0x5621ddd83fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 shutdown_connections 2024-05-14T17:06:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 1 -- 172.21.15.99:0/2632523375 wait complete. 2024-05-14T17:06:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac319b740 20 client.15609 trim_cache size 0 max 16384 2024-05-14T17:06:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:06:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:06:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4aa40b2a10 msgr2=0x7f4aa40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4aa40b2a10 0x7f4aa40b4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5621ddd7be90 tx=0x7f4ab400b390 comp rx=0 tx=0).stop 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 msgr2=0x7f4ab81a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 0x7f4ab81a4fd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009fd0 tx=0x7f4ab000b260 comp rx=0 tx=0).stop 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 shutdown_connections 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4aa40b2a10 0x7f4aa40b4ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab81068f0 0x7f4ab81a4fd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.872+0000 7f4ac047e640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab8102660 0x7f4ab819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.873+0000 7f4ac047e640 1 --2- 172.21.15.99:0/3947860900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ab8101ca0 0x7f4ab8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.873+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 >> 172.21.15.99:0/3947860900 conn(0x7f4ab8077750 msgr2=0x7f4ab80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.873+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 shutdown_connections 2024-05-14T17:06:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:18.873+0000 7f4ac047e640 1 -- 172.21.15.99:0/3947860900 wait complete. 2024-05-14T17:06:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:18 smithi099 ceph-mon[31851]: pgmap v83: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 20 KiB/s wr, 41 op/s 2024-05-14T17:06:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:18 smithi193 ceph-mon[35962]: pgmap v83: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 20 KiB/s wr, 41 op/s 2024-05-14T17:06:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:18 smithi150 ceph-mon[36204]: pgmap v83: 97 pgs: 97 active+clean; 13 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 20 KiB/s wr, 41 op/s 2024-05-14T17:06:19.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 4.38 MB/sec warmup 31 sec latency 23296.402 ms 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 -- 172.21.15.99:0/3053954502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 msgr2=0x7fdaa806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/3053954502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa806c670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda98009990 tx=0x7fda98009ca0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 -- 172.21.15.99:0/3053954502 shutdown_connections 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/3053954502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8105fb0 0x7fdaa8106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/3053954502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8108490 0x7fdaa806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/3053954502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa806c670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 -- 172.21.15.99:0/3053954502 >> 172.21.15.99:0/3053954502 conn(0x7fdaa80784b0 msgr2=0x7fdaa80788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 -- 172.21.15.99:0/3053954502 shutdown_connections 2024-05-14T17:06:20.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.056+0000 7fdaafcf3700 1 -- 172.21.15.99:0/3053954502 wait complete. 2024-05-14T17:06:20.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 Processor -- start 2024-05-14T17:06:20.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 -- start start 2024-05-14T17:06:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa81a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8105fb0 0x7fdaa81a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8108490 0x7fdaa81a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8119d80 con 0x7fdaa806c1f0 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8119c00 con 0x7fdaa8105fb0 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaafcf3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8119f00 con 0x7fdaa8108490 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa81a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa81a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43222/0 (socket says 172.21.15.99:43222) 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 -- 172.21.15.99:0/4121129171 learned_addr learned my addr 172.21.15.99:0/4121129171 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaae290700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8108490 0x7fdaa81a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaad28e700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8105fb0 0x7fdaa81a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 -- 172.21.15.99:0/4121129171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8108490 msgr2=0x7fdaa81a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8108490 0x7fdaa81a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 -- 172.21.15.99:0/4121129171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8105fb0 msgr2=0x7fdaa81a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8105fb0 0x7fdaa81a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 -- 172.21.15.99:0/4121129171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa81a52e0 con 0x7fdaa806c1f0 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaae290700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8108490 0x7fdaa81a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:20.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaad28e700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8105fb0 0x7fdaa81a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.057+0000 7fdaada8f700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa81a02d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400ba50 tx=0x7fdaa400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.058+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4019070 con 0x7fdaa806c1f0 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.058+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa4014440 con 0x7fdaa806c1f0 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.058+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa81a55d0 con 0x7fdaa806c1f0 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.058+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40075d0 con 0x7fdaa806c1f0 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.058+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa81ace50 con 0x7fdaa806c1f0 2024-05-14T17:06:20.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.059+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa8102a90 con 0x7fdaa806c1f0 2024-05-14T17:06:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.060+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdaa401c070 con 0x7fdaa806c1f0 2024-05-14T17:06:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.061+0000 7fda9effd700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fda940b29f0 0x7fda940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.061+0000 7fdaad28e700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fda940b29f0 0x7fda940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.061+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdaa40e1e40 con 0x7fdaa806c1f0 2024-05-14T17:06:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.061+0000 7fdaad28e700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fda940b29f0 0x7fda940b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b5c0 tx=0x7fda98005fb0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:20.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.064+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdaa40a9960 con 0x7fdaa806c1f0 2024-05-14T17:06:20.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.314+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa806a230 con 0x7fdaa806c1f0 2024-05-14T17:06:20.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.315+0000 7fda9effd700 1 -- 172.21.15.99:0/4121129171 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7fdaa806a230 con 0x7fdaa806c1f0 2024-05-14T17:06:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:06:20.315 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fda940b29f0 msgr2=0x7fda940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fda940b29f0 0x7fda940b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda9800b5c0 tx=0x7fda98005fb0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 msgr2=0x7fdaa81a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa81a02d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400ba50 tx=0x7fdaa400be10 comp rx=0 tx=0).stop 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 shutdown_connections 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fda940b29f0 0x7fda940b4eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdaa8108490 0x7fdaa81a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaa806c1f0 0x7fdaa81a02d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 --2- 172.21.15.99:0/4121129171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8105fb0 0x7fdaa81a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:20.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.317+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 >> 172.21.15.99:0/4121129171 conn(0x7fdaa80784b0 msgr2=0x7fdaa8104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:20.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.318+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 shutdown_connections 2024-05-14T17:06:20.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:20.318+0000 7fdaafcf3700 1 -- 172.21.15.99:0/4121129171 wait complete. 2024-05-14T17:06:20.318 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:06:20.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 4.24 MB/sec warmup 32 sec latency 24296.503 ms 2024-05-14T17:06:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:20 smithi099 ceph-mon[31851]: pgmap v84: 97 pgs: 97 active+clean; 25 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.7 MiB/s wr, 99 op/s 2024-05-14T17:06:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:20 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4121129171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:20 smithi193 ceph-mon[35962]: pgmap v84: 97 pgs: 97 active+clean; 25 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.7 MiB/s wr, 99 op/s 2024-05-14T17:06:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:20 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4121129171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:21.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:20 smithi150 ceph-mon[36204]: pgmap v84: 97 pgs: 97 active+clean; 25 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.7 MiB/s wr, 99 op/s 2024-05-14T17:06:21.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:20 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4121129171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:21.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 4.11 MB/sec warmup 33 sec latency 25296.570 ms 2024-05-14T17:06:22.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 3.99 MB/sec warmup 34 sec latency 26296.673 ms 2024-05-14T17:06:22.991 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id b889149b --release --await' 2024-05-14T17:06:22.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id b889149b --release --await 2024-05-14T17:06:23.236 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:22 smithi099 ceph-mon[31851]: pgmap v85: 97 pgs: 97 active+clean; 25 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.7 MiB/s wr, 60 op/s 2024-05-14T17:06:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:22 smithi193 ceph-mon[35962]: pgmap v85: 97 pgs: 97 active+clean; 25 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.7 MiB/s wr, 60 op/s 2024-05-14T17:06:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:22 smithi150 ceph-mon[36204]: pgmap v85: 97 pgs: 97 active+clean; 25 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.7 MiB/s wr, 60 op/s 2024-05-14T17:06:23.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 3.88 MB/sec warmup 35 sec latency 27296.750 ms 2024-05-14T17:06:24.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 21933 3.77 MB/sec warmup 36 sec latency 28296.830 ms 2024-05-14T17:06:24.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 -- 172.21.15.99:0/2412051820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701037f0 msgr2=0x7f9a70103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2412051820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701037f0 0x7f9a70103bd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0099c0 tx=0x7f9a6c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 -- 172.21.15.99:0/2412051820 shutdown_connections 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2412051820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a701088f0 0x7f9a70110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2412051820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701037f0 0x7f9a70103bd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2412051820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701041a0 0x7f9a701081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 -- 172.21.15.99:0/2412051820 >> 172.21.15.99:0/2412051820 conn(0x7f9a7007a6a0 msgr2=0x7f9a7007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 -- 172.21.15.99:0/2412051820 shutdown_connections 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.884+0000 7f9a7856f700 1 -- 172.21.15.99:0/2412051820 wait complete. 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 Processor -- start 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 -- start start 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701037f0 0x7f9a7019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 0x7f9a7019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a701088f0 0x7f9a70199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701136c0 con 0x7f9a701041a0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70113540 con 0x7f9a701088f0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7856f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70113840 con 0x7f9a701037f0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 0x7f9a7019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 0x7f9a7019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56648/0 (socket says 172.21.15.99:56648) 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 -- 172.21.15.99:0/2723302313 learned_addr learned my addr 172.21.15.99:0/2723302313 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a701088f0 0x7f9a70199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701037f0 0x7f9a7019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 -- 172.21.15.99:0/2723302313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701037f0 msgr2=0x7f9a7019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701037f0 0x7f9a7019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 -- 172.21.15.99:0/2723302313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a701088f0 msgr2=0x7f9a70199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a701088f0 0x7f9a70199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 -- 172.21.15.99:0/2723302313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7019a180 con 0x7f9a701041a0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701037f0 0x7f9a7019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.885+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 0x7f9a7019fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000d980 tx=0x7f9a6000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.886+0000 7f9a677fe700 1 -- 172.21.15.99:0/2723302313 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60016070 con 0x7f9a701041a0 2024-05-14T17:06:24.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.886+0000 7f9a677fe700 1 -- 172.21.15.99:0/2723302313 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a600044d0 con 0x7f9a701041a0 2024-05-14T17:06:24.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.886+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7019a470 con 0x7f9a701041a0 2024-05-14T17:06:24.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.886+0000 7f9a677fe700 1 -- 172.21.15.99:0/2723302313 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60004d00 con 0x7f9a701041a0 2024-05-14T17:06:24.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.886+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701a66e0 con 0x7f9a701041a0 2024-05-14T17:06:24.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.889+0000 7f9a677fe700 1 -- 172.21.15.99:0/2723302313 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9a60004020 con 0x7f9a701041a0 2024-05-14T17:06:24.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.889+0000 7f9a677fe700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9a5c0b29f0 0x7f9a5c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9a5c0b29f0 0x7f9a5c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a677fe700 1 -- 172.21.15.99:0/2723302313 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f9a600e2590 con 0x7f9a701041a0 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 Processor -- start 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 -- start start 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9a5c0b29f0 0x7f9a5c0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c000c00 tx=0x7f9a6c005940 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42a7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bcf4204e70 con 0x55bcf42a72c0 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bcf41f4a10 con 0x55bcf42ab420 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bcf41f8a50 con 0x55bcf42abd40 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.890+0000 7f9a76b0c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56650/0 (socket says 172.21.15.99:56650) 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3432400774 learned_addr learned my addr 172.21.15.99:0/3432400774 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a7630b700 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3432400774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 msgr2=0x55bcf42a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3432400774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 msgr2=0x55bcf42ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3432400774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf419e0b0 con 0x55bcf42a72c0 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42a7720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009d20 tx=0x7f9a680077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a861b22eaafa4d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a64ff9700 1 -- 172.21.15.99:0/3432400774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68004bb0 con 0x55bcf42a72c0 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a64ff9700 1 -- 172.21.15.99:0/3432400774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68004d50 con 0x55bcf42a72c0 2024-05-14T17:06:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.891+0000 7f9a64ff9700 1 -- 172.21.15.99:0/3432400774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68014670 con 0x55bcf42a72c0 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3432400774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 msgr2=0x55bcf42a7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42a7720 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009d20 tx=0x7f9a680077e0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3432400774 shutdown_connections 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42a7720 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3432400774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3432400774 >> 172.21.15.99:0/3432400774 conn(0x55bcf41d3eb0 msgr2=0x55bcf42a14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3432400774 shutdown_connections 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3432400774 wait complete. 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.892+0000 7f9a82e05b80 1 -- start start 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 Processor -- start 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bcf40fb0a0 con 0x55bcf42a72c0 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bcf415cea0 con 0x55bcf42ab420 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bcf42b3880 con 0x55bcf42abd40 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a82e05b80 10 client.0 _ll_register_callbacks cb 0x55bcf418fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56652/0 (socket says 172.21.15.99:56652) 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 -- 172.21.15.99:0/921469961 learned_addr learned my addr 172.21.15.99:0/921469961 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 -- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 msgr2=0x55bcf42b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 -- 172.21.15.99:0/921469961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 msgr2=0x55bcf42afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 -- 172.21.15.99:0/921469961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf41449d0 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a7630b700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42b2a90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0098e0 tx=0x7f9a6c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28406ec29b88f1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.893+0000 7f9a59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a59ffb700 1 -- 172.21.15.99:0/921469961 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c035060 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a59ffb700 1 -- 172.21.15.99:0/921469961 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f9a6c00fe00 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a68004d50 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a59ffb700 1 -- 172.21.15.99:0/921469961 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c003860 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 10 client.15627 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcf41ae860 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a59ffb700 1 -- 172.21.15.99:0/921469961 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68004d50 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a59ffb700 1 -- 172.21.15.99:0/921469961 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9a6c030460 con 0x55bcf42a72c0 2024-05-14T17:06:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 10 client.15627 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 20 client.15627 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf42c03c0 0x55bcf42c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 4 client.15627 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.894+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcf4206dc0 con 0x55bcf42c03c0 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.895+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf42c03c0 0x55bcf42c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.895+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf42c03c0 0x55bcf42c27b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254cb41b8856cc02 server_cookie=929cf72cb3dd9a4b in_seq=0 out_seq=0 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.895+0000 7f9a59ffb700 10 client.15627 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a59ffb700 1 -- 172.21.15.99:0/921469961 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcf4206dc0 con 0x55bcf42c03c0 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a59ffb700 10 client.15627 handle_command_reply: tid=0 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 client.15627 shutdown 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 msgr2=0x55bcf42b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42b2a90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0098e0 tx=0x7f9a6c01a050 comp rx=0 tx=0).stop 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 shutdown_connections 2024-05-14T17:06:24.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42abd40 0x55bcf42b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf42c03c0 0x55bcf42c27b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf42b2a90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/921469961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a59ffb700 0 client.15627 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 >> 172.21.15.99:0/921469961 conn(0x55bcf41d3eb0 msgr2=0x55bcf429bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 shutdown_connections 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 1 -- 172.21.15.99:0/921469961 wait complete. 2024-05-14T17:06:24.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.896+0000 7f9a82e05b80 20 client.15627 trim_cache size 0 max 16384 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.915+0000 7f9a82e05b80 1 Processor -- start 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 -- start start 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf429c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42ab420 0x55bcf42a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42abd40 0x55bcf42a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701b0520 con 0x55bcf42a72c0 2024-05-14T17:06:24.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701addb0 con 0x55bcf42abd40 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701abc00 con 0x55bcf42ab420 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf429c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf429c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56662/0 (socket says 172.21.15.99:56662) 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 -- 172.21.15.99:0/2616486532 learned_addr learned my addr 172.21.15.99:0/2616486532 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42ab420 0x55bcf42a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42abd40 0x55bcf42a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 -- 172.21.15.99:0/2616486532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42ab420 msgr2=0x55bcf42a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42ab420 0x55bcf42a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 -- 172.21.15.99:0/2616486532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42abd40 msgr2=0x55bcf42a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42abd40 0x55bcf42a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.916+0000 7f9a7630b700 1 -- 172.21.15.99:0/2616486532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf42a6fa0 con 0x55bcf42a72c0 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.917+0000 7f9a7630b700 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf429c0d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b580 tx=0x7f9a6c030670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee788e3a9fe1583 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.917+0000 7f9a5bfff700 1 -- 172.21.15.99:0/2616486532 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c031e60 con 0x55bcf42a72c0 2024-05-14T17:06:24.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.917+0000 7f9a5bfff700 1 -- 172.21.15.99:0/2616486532 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6c00fe00 con 0x55bcf42a72c0 2024-05-14T17:06:24.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.917+0000 7f9a5bfff700 1 -- 172.21.15.99:0/2616486532 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c035040 con 0x55bcf42a72c0 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- 172.21.15.99:0/2616486532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 msgr2=0x55bcf429c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf429c0d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b580 tx=0x7f9a6c030670 comp rx=0 tx=0).stop 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- 172.21.15.99:0/2616486532 shutdown_connections 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42abd40 0x55bcf42a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42a72c0 0x55bcf429c0d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/2616486532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42ab420 0x55bcf42a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- 172.21.15.99:0/2616486532 >> 172.21.15.99:0/2616486532 conn(0x55bcf41d3eb0 msgr2=0x55bcf42b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- 172.21.15.99:0/2616486532 shutdown_connections 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- 172.21.15.99:0/2616486532 wait complete. 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- start start 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 Processor -- start 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42a72c0 0x55bcf42b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 0x55bcf432ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701ac270 con 0x55bcf42abd40 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701ad6e0 con 0x55bcf42ab420 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.918+0000 7f9a82e05b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701ac940 con 0x55bcf42a72c0 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a82e05b80 10 client.0 _ll_register_callbacks cb 0x55bcf418fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 0x55bcf432ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 0x55bcf432ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56676/0 (socket says 172.21.15.99:56676) 2024-05-14T17:06:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3643732764 learned_addr learned my addr 172.21.15.99:0/3643732764 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a75b0a700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a7630b700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42a72c0 0x55bcf42b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42a72c0 msgr2=0x55bcf42b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42a72c0 0x55bcf42b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3643732764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 msgr2=0x55bcf42b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 -- 172.21.15.99:0/3643732764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf432b1b0 con 0x55bcf42abd40 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a76b0c700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 0x55bcf432ac70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9a680092f0 tx=0x7f9a68005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8772578eaa745b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a59ffb700 1 -- 172.21.15.99:0/3643732764 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68023070 con 0x55bcf42abd40 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a6c00fe00 con 0x55bcf42abd40 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.919+0000 7f9a59ffb700 1 -- 172.21.15.99:0/3643732764 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f9a680056a0 con 0x55bcf42abd40 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a59ffb700 1 -- 172.21.15.99:0/3643732764 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9a680058e0 con 0x55bcf42abd40 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 10 client.15633 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcf432c3d0 con 0x55bcf42abd40 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a59ffb700 1 -- 172.21.15.99:0/3643732764 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6c00fe00 con 0x55bcf42abd40 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a59ffb700 1 -- 172.21.15.99:0/3643732764 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9a68028b50 con 0x55bcf42abd40 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 10 client.15633 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 20 client.15633 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf432c830 0x55bcf432ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 4 client.15633 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b889149b", "release": true, "await": true}] 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b889149b", "release": true, "await": true}) -- 0x55bcf42005a0 con 0x55bcf432c830 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.920+0000 7f9a7630b700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf432c830 0x55bcf432ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.921+0000 7f9a59ffb700 10 client.15633 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.921+0000 7f9a7630b700 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf432c830 0x55bcf432ec50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4753b45073b2d6c server_cookie=1f39e378a6230da9 in_seq=0 out_seq=0 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a59ffb700 1 -- 172.21.15.99:0/3643732764 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55bcf42005a0 con 0x55bcf432c830 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a59ffb700 10 client.15633 handle_command_reply: tid=0 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 client.15633 shutdown 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 msgr2=0x55bcf432ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 0x55bcf432ac70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9a680092f0 tx=0x7f9a68005270 comp rx=0 tx=0).stop 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 shutdown_connections 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bcf432c830 0x55bcf432ec50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bcf42abd40 0x55bcf432ac70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcf42a72c0 0x55bcf42b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a59ffb700 0 client.15633 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 --2- 172.21.15.99:0/3643732764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bcf42ab420 0x55bcf42b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.930+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 >> 172.21.15.99:0/3643732764 conn(0x55bcf41d3eb0 msgr2=0x55bcf42b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 shutdown_connections 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a82e05b80 1 -- 172.21.15.99:0/3643732764 wait complete. 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a82e05b80 20 client.15633 trim_cache size 0 max 16384 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:06:24.931 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 8, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "b889149b": { 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 8, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 146.7, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:06:24.932 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:24.933 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9a5c0b29f0 msgr2=0x7f9a5c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9a5c0b29f0 0x7f9a5c0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c000c00 tx=0x7f9a6c005940 comp rx=0 tx=0).stop 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 msgr2=0x7f9a7019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 0x7f9a7019fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000d980 tx=0x7f9a6000dd40 comp rx=0 tx=0).stop 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 shutdown_connections 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9a5c0b29f0 0x7f9a5c0b4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a701088f0 0x7f9a70199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a701037f0 0x7f9a7019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 --2- 172.21.15.99:0/2723302313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a701041a0 0x7f9a7019fd50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 >> 172.21.15.99:0/2723302313 conn(0x7f9a7007a6a0 msgr2=0x7f9a70109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 shutdown_connections 2024-05-14T17:06:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:24.931+0000 7f9a7856f700 1 -- 172.21.15.99:0/2723302313 wait complete. 2024-05-14T17:06:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:24 smithi099 ceph-mon[31851]: pgmap v86: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.7 MiB/s wr, 69 op/s 2024-05-14T17:06:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:24 smithi193 ceph-mon[35962]: pgmap v86: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.7 MiB/s wr, 69 op/s 2024-05-14T17:06:25.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:24 smithi150 ceph-mon[36204]: pgmap v86: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.7 MiB/s wr, 69 op/s 2024-05-14T17:06:25.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 23767 3.86 MB/sec warmup 37 sec latency 28712.407 ms 2024-05-14T17:06:25.740 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b889149b', total seconds elapsed: 34.3 2024-05-14T17:06:25.741 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.7 2024-05-14T17:06:25.741 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:06:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:26.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 26106 4.13 MB/sec warmup 38 sec latency 18.932 ms 2024-05-14T17:06:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:26 smithi099 ceph-mon[31851]: pgmap v87: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 67 op/s 2024-05-14T17:06:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:26 smithi193 ceph-mon[35962]: pgmap v87: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 67 op/s 2024-05-14T17:06:27.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:26 smithi150 ceph-mon[36204]: pgmap v87: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 67 op/s 2024-05-14T17:06:27.518 INFO:tasks.workunit.client.0.smithi099.stdout: 1 28968 4.46 MB/sec warmup 39 sec latency 15.464 ms 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 -- 172.21.15.99:0/1770582173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 msgr2=0x7fe50010eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 --2- 172.21.15.99:0/1770582173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe50010eb20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800b0a0 tx=0x7fe4f800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 -- 172.21.15.99:0/1770582173 shutdown_connections 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 --2- 172.21.15.99:0/1770582173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe50010eb20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 --2- 172.21.15.99:0/1770582173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe500102140 0x7fe500102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 --2- 172.21.15.99:0/1770582173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500102af0 0x7fe5001068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 -- 172.21.15.99:0/1770582173 >> 172.21.15.99:0/1770582173 conn(0x7fe500077760 msgr2=0x7fe500077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:27.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 -- 172.21.15.99:0/1770582173 shutdown_connections 2024-05-14T17:06:27.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 -- 172.21.15.99:0/1770582173 wait complete. 2024-05-14T17:06:27.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.653+0000 7fe506046700 1 Processor -- start 2024-05-14T17:06:27.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 -- start start 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe500102140 0x7fe50019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500102af0 0x7fe50019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe500199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001137c0 con 0x7fe500106fe0 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe500113640 con 0x7fe500102140 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe506046700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe500113940 con 0x7fe500102af0 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe500199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe500199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56698/0 (socket says 172.21.15.99:56698) 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe500102140 0x7fe50019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 -- 172.21.15.99:0/2127702141 learned_addr learned my addr 172.21.15.99:0/2127702141 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4feffd700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500102af0 0x7fe50019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 -- 172.21.15.99:0/2127702141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500102af0 msgr2=0x7fe50019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500102af0 0x7fe50019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 -- 172.21.15.99:0/2127702141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe500102140 msgr2=0x7fe50019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:27.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe500102140 0x7fe50019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 -- 172.21.15.99:0/2127702141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50019a180 con 0x7fe500106fe0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.654+0000 7fe4fffff700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe500199a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800b070 tx=0x7fe4f8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.655+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8021070 con 0x7fe500106fe0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.655+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f8007e30 con 0x7fe500106fe0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.655+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50019a410 con 0x7fe500106fe0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.655+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f80046d0 con 0x7fe500106fe0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.655+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a6730 con 0x7fe500106fe0 2024-05-14T17:06:27.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.656+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50006a230 con 0x7fe500106fe0 2024-05-14T17:06:27.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.657+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe4f8007980 con 0x7fe500106fe0 2024-05-14T17:06:27.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.658+0000 7fe4fcff9700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4e80b2a70 0x7fe4e80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:27.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.658+0000 7fe4ff7fe700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4e80b2a70 0x7fe4e80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.658+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe4f800f020 con 0x7fe500106fe0 2024-05-14T17:06:27.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.658+0000 7fe4ff7fe700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4e80b2a70 0x7fe4e80b4f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe50019ab60 tx=0x7fe4f000b410 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:27.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.662+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4f802a020 con 0x7fe500106fe0 2024-05-14T17:06:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.937+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe50019af50 con 0x7fe500106fe0 2024-05-14T17:06:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.938+0000 7fe4fcff9700 1 -- 172.21.15.99:0/2127702141 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11873 (secure 0 0 0) 0x7fe4f802e3e0 con 0x7fe500106fe0 2024-05-14T17:06:27.938 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:06:27.938 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:03:41.795311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.940+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4e80b2a70 msgr2=0x7fe4e80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.940+0000 7fe506046700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4e80b2a70 0x7fe4e80b4f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe50019ab60 tx=0x7fe4f000b410 comp rx=0 tx=0).stop 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.940+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 msgr2=0x7fe500199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.940+0000 7fe506046700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe500199a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800b070 tx=0x7fe4f8004cf0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 shutdown_connections 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4e80b2a70 0x7fe4e80b4f30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe500106fe0 0x7fe500199a20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe500102140 0x7fe50019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 --2- 172.21.15.99:0/2127702141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500102af0 0x7fe50019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 >> 172.21.15.99:0/2127702141 conn(0x7fe500077760 msgr2=0x7fe5000ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 shutdown_connections 2024-05-14T17:06:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:27.941+0000 7fe506046700 1 -- 172.21.15.99:0/2127702141 wait complete. 2024-05-14T17:06:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-05-14T17:06:28.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:28 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2127702141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:28.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:28 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2127702141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:28.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:28 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2127702141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:28.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 32103 4.79 MB/sec warmup 40 sec latency 13.711 ms 2024-05-14T17:06:28.684 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0' 2024-05-14T17:06:28.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0 2024-05-14T17:06:28.934 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:29 smithi099 ceph-mon[31851]: pgmap v88: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 67 op/s 2024-05-14T17:06:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:29 smithi193 ceph-mon[35962]: pgmap v88: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 67 op/s 2024-05-14T17:06:29.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:29 smithi150 ceph-mon[36204]: pgmap v88: 97 pgs: 97 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 67 op/s 2024-05-14T17:06:29.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 34566 4.91 MB/sec warmup 41 sec latency 20.701 ms 2024-05-14T17:06:30.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 36759 5.10 MB/sec warmup 42 sec latency 21.655 ms 2024-05-14T17:06:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.564+0000 7f0332a45700 1 -- 172.21.15.99:0/2295157979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103590 msgr2=0x7f032c103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.564+0000 7f0332a45700 1 --2- 172.21.15.99:0/2295157979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103590 0x7f032c103a10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f03200099c0 tx=0x7f0320009cd0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 -- 172.21.15.99:0/2295157979 shutdown_connections 2024-05-14T17:06:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 --2- 172.21.15.99:0/2295157979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f032c103f50 0x7f032c110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 --2- 172.21.15.99:0/2295157979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c109640 0x7f032c109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 --2- 172.21.15.99:0/2295157979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103590 0x7f032c103a10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 -- 172.21.15.99:0/2295157979 >> 172.21.15.99:0/2295157979 conn(0x7f032c0ff020 msgr2=0x7f032c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:30.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 -- 172.21.15.99:0/2295157979 shutdown_connections 2024-05-14T17:06:30.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 -- 172.21.15.99:0/2295157979 wait complete. 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 Processor -- start 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.565+0000 7f0332a45700 1 -- start start 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0332a45700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f032c103590 0x7f032c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0332a45700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 0x7f032c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0332a45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c109640 0x7f032c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0332a45700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f032c077180 con 0x7f032c103f50 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0332a45700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f032c077000 con 0x7f032c103590 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0332a45700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f032c077300 con 0x7f032c109640 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 0x7f032c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 0x7f032c1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56716/0 (socket says 172.21.15.99:56716) 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 -- 172.21.15.99:0/4078367297 learned_addr learned my addr 172.21.15.99:0/4078367297 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f0330fe2700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c109640 0x7f032c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032bfff700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f032c103590 0x7f032c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 -- 172.21.15.99:0/4078367297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c109640 msgr2=0x7f032c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c109640 0x7f032c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 -- 172.21.15.99:0/4078367297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f032c103590 msgr2=0x7f032c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f032c103590 0x7f032c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.566+0000 7f032b7fe700 1 -- 172.21.15.99:0/4078367297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c19c450 con 0x7f032c103f50 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.567+0000 7f032b7fe700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 0x7f032c1a1f90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0320006010 tx=0x7f03200055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.567+0000 7f03297fa700 1 -- 172.21.15.99:0/4078367297 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320026070 con 0x7f032c103f50 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.567+0000 7f03297fa700 1 -- 172.21.15.99:0/4078367297 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032000f460 con 0x7f032c103f50 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.567+0000 7f03297fa700 1 -- 172.21.15.99:0/4078367297 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320009e60 con 0x7f032c103f50 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.567+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c19c6e0 con 0x7f032c103f50 2024-05-14T17:06:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.567+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1a89d0 con 0x7f032c103f50 2024-05-14T17:06:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.570+0000 7f03297fa700 1 -- 172.21.15.99:0/4078367297 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0320029070 con 0x7f032c103f50 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.571+0000 7f03297fa700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f03140b2b20 0x7f03140b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.571+0000 7f032bfff700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f03140b2b20 0x7f03140b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.571+0000 7f03297fa700 1 -- 172.21.15.99:0/4078367297 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f03200e2d30 con 0x7f032c103f50 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.571+0000 7f033d2dbb80 1 Processor -- start 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 -- start start 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f03140b2b20 0x7f03140b4fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f031c007be0 tx=0x7f031c0073d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 0x55e445652890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44564dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e44564e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e4455abf00 con 0x55e4456524b0 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e44559baa0 con 0x55e44564e350 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e44559fae0 con 0x55e445652dd0 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 0x55e445652890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 0x55e445652890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56724/0 (socket says 172.21.15.99:56724) 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 -- 172.21.15.99:0/4128773737 learned_addr learned my addr 172.21.15.99:0/4128773737 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44564dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f0330fe2700 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e44564e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 -- 172.21.15.99:0/4128773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 msgr2=0x55e44564dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44564dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 -- 172.21.15.99:0/4128773737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 msgr2=0x55e44564e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e44564e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 -- 172.21.15.99:0/4128773737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e445545140 con 0x55e4456524b0 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032b7fe700 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44564dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:06:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.572+0000 7f032bfff700 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 0x55e445652890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f031c010e80 tx=0x7f031c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9053626fe37c23f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f03127fc700 1 -- 172.21.15.99:0/4128773737 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c024070 con 0x55e4456524b0 2024-05-14T17:06:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f03127fc700 1 -- 172.21.15.99:0/4128773737 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c015070 con 0x55e4456524b0 2024-05-14T17:06:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f03127fc700 1 -- 172.21.15.99:0/4128773737 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0118c0 con 0x55e4456524b0 2024-05-14T17:06:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 -- 172.21.15.99:0/4128773737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 msgr2=0x55e445652890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 0x55e445652890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f031c010e80 tx=0x7f031c013000 comp rx=0 tx=0).stop 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 -- 172.21.15.99:0/4128773737 shutdown_connections 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e44564e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4456524b0 0x55e445652890 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/4128773737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44564dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 -- 172.21.15.99:0/4128773737 >> 172.21.15.99:0/4128773737 conn(0x55e44557af40 msgr2=0x55e445648530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 -- 172.21.15.99:0/4128773737 shutdown_connections 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.573+0000 7f033d2dbb80 1 -- 172.21.15.99:0/4128773737 wait complete. 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 -- start start 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 Processor -- start 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e44564e350 0x55e445659bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e445656f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e445657490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e4454a2130 con 0x55e445652dd0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e445503f30 con 0x55e4456524b0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e44565a910 con 0x55e44564e350 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f033d2dbb80 10 client.0 _ll_register_callbacks cb 0x55e445536ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f0330fe2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e445657490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f032bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e44564e350 0x55e445659bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f0330fe2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e445657490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56740/0 (socket says 172.21.15.99:56740) 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.574+0000 7f0330fe2700 1 -- 172.21.15.99:0/790706226 learned_addr learned my addr 172.21.15.99:0/790706226 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f032b7fe700 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e445656f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f0330fe2700 1 -- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e44564e350 msgr2=0x55e445659bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f0330fe2700 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e44564e350 0x55e445659bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f0330fe2700 1 -- 172.21.15.99:0/790706226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 msgr2=0x55e445656f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f0330fe2700 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e445656f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f0330fe2700 1 -- 172.21.15.99:0/790706226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4454eba60 con 0x55e445652dd0 2024-05-14T17:06:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f0330fe2700 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e445657490 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0324011f60 tx=0x7f0324014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36b2fb35667bbae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f030b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f030b7fe700 1 -- 172.21.15.99:0/790706226 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324014ba0 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f030b7fe700 1 -- 172.21.15.99:0/790706226 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f0324014d40 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031c015070 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f030b7fe700 1 -- 172.21.15.99:0/790706226 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0324003a00 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f033d2dbb80 10 client.15651 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4455558f0 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.575+0000 7f030b7fe700 1 -- 172.21.15.99:0/790706226 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c015070 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f030b7fe700 1 -- 172.21.15.99:0/790706226 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0324007500 con 0x55e445652dd0 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f033d2dbb80 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f033d2dbb80 10 client.15651 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f033d2dbb80 20 client.15651 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456674f0 0x55e4456698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f033d2dbb80 4 client.15651 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4455ade50 con 0x55e4456674f0 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f032bfff700 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456674f0 0x55e4456698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f032bfff700 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456674f0 0x55e4456698e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e17d84aae92f5237 server_cookie=205a2d857f5d1d93 in_seq=0 out_seq=0 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.576+0000 7f030b7fe700 10 client.15651 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f030b7fe700 1 -- 172.21.15.99:0/790706226 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4455ade50 con 0x55e4456674f0 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f030b7fe700 10 client.15651 handle_command_reply: tid=0 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 client.15651 shutdown 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 msgr2=0x55e445657490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e445657490 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0324011f60 tx=0x7f0324014600 comp rx=0 tx=0).stop 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 shutdown_connections 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456674f0 0x55e4456698e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e445657490 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e44564e350 0x55e445659bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/790706226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e445656f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 >> 172.21.15.99:0/790706226 conn(0x55e44557af40 msgr2=0x55e445643050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:30.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f030b7fe700 0 client.15651 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:30.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 shutdown_connections 2024-05-14T17:06:30.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.577+0000 7f033d2dbb80 1 -- 172.21.15.99:0/790706226 wait complete. 2024-05-14T17:06:30.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.578+0000 7f033d2dbb80 20 client.15651 trim_cache size 0 max 16384 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.598+0000 7f033d2dbb80 1 Processor -- start 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.598+0000 7f033d2dbb80 1 -- start start 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.598+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e445643160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.598+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 0x55e445651880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.598+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e44566f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b2610 con 0x55e445652dd0 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1afea0 con 0x55e44564e350 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1add10 con 0x55e4456524b0 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f0330fe2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e44566f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 0x55e445651880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 0x55e445651880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:47976/0 (socket says 172.21.15.99:47976) 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f0330fe2700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e44566f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56756/0 (socket says 172.21.15.99:56756) 2024-05-14T17:06:30.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032bfff700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e445643160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 -- 172.21.15.99:0/1794178729 learned_addr learned my addr 172.21.15.99:0/1794178729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 -- 172.21.15.99:0/1794178729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 msgr2=0x55e445643160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e445643160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 -- 172.21.15.99:0/1794178729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 msgr2=0x55e44566f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e44566f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 -- 172.21.15.99:0/1794178729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4456554c0 con 0x55e4456524b0 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032bfff700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e445643160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f0330fe2700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e44566f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f032b7fe700 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 0x55e445651880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0320009990 tx=0x7f03200092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c5642f6f56aceb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f0311ffb700 1 -- 172.21.15.99:0/1794178729 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200ae480 con 0x55e4456524b0 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f0311ffb700 1 -- 172.21.15.99:0/1794178729 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03200b2ea0 con 0x55e4456524b0 2024-05-14T17:06:30.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.599+0000 7f0311ffb700 1 -- 172.21.15.99:0/1794178729 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200aa330 con 0x55e4456524b0 2024-05-14T17:06:30.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 -- 172.21.15.99:0/1794178729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 msgr2=0x55e445651880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 0x55e445651880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0320009990 tx=0x7f03200092b0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 -- 172.21.15.99:0/1794178729 shutdown_connections 2024-05-14T17:06:30.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e445652dd0 0x55e44566f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e44564e350 0x55e445643160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/1794178729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4456524b0 0x55e445651880 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 -- 172.21.15.99:0/1794178729 >> 172.21.15.99:0/1794178729 conn(0x55e44557af40 msgr2=0x55e445651170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 -- 172.21.15.99:0/1794178729 shutdown_connections 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 -- 172.21.15.99:0/1794178729 wait complete. 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.600+0000 7f033d2dbb80 1 -- start start 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 Processor -- start 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 0x55e445660110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e44565c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44565c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1ae360 con 0x55e44564e350 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1af7d0 con 0x55e4456524b0 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1aea30 con 0x55e445652dd0 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f033d2dbb80 10 client.0 _ll_register_callbacks cb 0x55e445536ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:30.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 0x55e445660110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 0x55e445660110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56762/0 (socket says 172.21.15.99:56762) 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 -- 172.21.15.99:0/2196973914 learned_addr learned my addr 172.21.15.99:0/2196973914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f0330fe2700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44565c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032b7fe700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e44565c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 -- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 msgr2=0x55e44565c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44565c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 -- 172.21.15.99:0/2196973914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 msgr2=0x55e44565c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.601+0000 7f032bfff700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e44565c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f032bfff700 1 -- 172.21.15.99:0/2196973914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44565d220 con 0x55e44564e350 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f032bfff700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 0x55e445660110 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f031c0074d0 tx=0x7f031c010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b72c6bfeae37a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f030b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f030b7fe700 1 -- 172.21.15.99:0/2196973914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c024040 con 0x55e44564e350 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f030b7fe700 1 -- 172.21.15.99:0/2196973914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f031c003a90 con 0x55e44564e350 2024-05-14T17:06:30.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03200b2ea0 con 0x55e44564e350 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f030b7fe700 1 -- 172.21.15.99:0/2196973914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f031c042050 con 0x55e44564e350 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f033d2dbb80 10 client.15657 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4456d2d70 con 0x55e44564e350 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f030b7fe700 1 -- 172.21.15.99:0/2196973914 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03200b2ea0 con 0x55e44564e350 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f030b7fe700 1 -- 172.21.15.99:0/2196973914 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f031c02a040 con 0x55e44564e350 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.602+0000 7f033d2dbb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f033d2dbb80 10 client.15657 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f033d2dbb80 20 client.15657 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456d3160 0x55e4456d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f033d2dbb80 4 client.15657 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}] 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}) -- 0x55e4455a7630 con 0x55e4456d3160 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f032b7fe700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456d3160 0x55e4456d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f030b7fe700 10 client.15657 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:30.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.603+0000 7f032b7fe700 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456d3160 0x55e4456d5550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396f347b82011c69 server_cookie=969e83bfa73fbb8b in_seq=0 out_seq=0 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f030b7fe700 1 -- 172.21.15.99:0/2196973914 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e4455a7630 con 0x55e4456d3160 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f030b7fe700 10 client.15657 handle_command_reply: tid=0 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 client.15657 shutdown 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 msgr2=0x55e445660110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 0x55e445660110 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f031c0074d0 tx=0x7f031c010f00 comp rx=0 tx=0).stop 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 shutdown_connections 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e445652dd0 0x55e44565c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e4456d3160 0x55e4456d5550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e44564e350 0x55e445660110 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f030b7fe700 0 client.15657 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 --2- 172.21.15.99:0/2196973914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4456524b0 0x55e44565c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 >> 172.21.15.99:0/2196973914 conn(0x55e44557af40 msgr2=0x55e445651a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:30.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 shutdown_connections 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 1 -- 172.21.15.99:0/2196973914 wait complete. 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.691+0000 7f033d2dbb80 20 client.15657 trim_cache size 0 max 16384 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 10, 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "380494ce": { 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 10, 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:06:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 146.7, 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f03140b2b20 msgr2=0x7f03140b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f03140b2b20 0x7f03140b4fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f031c007be0 tx=0x7f031c0073d0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 msgr2=0x7f032c1a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 0x7f032c1a1f90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0320006010 tx=0x7f03200055f0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 shutdown_connections 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f03140b2b20 0x7f03140b4fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c109640 0x7f032c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f032c103590 0x7f032c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 --2- 172.21.15.99:0/4078367297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f032c103f50 0x7f032c1a1f90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 >> 172.21.15.99:0/4078367297 conn(0x7f032c0ff020 msgr2=0x7f032c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 shutdown_connections 2024-05-14T17:06:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:30.692+0000 7f0332a45700 1 -- 172.21.15.99:0/4078367297 wait complete. 2024-05-14T17:06:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:31 smithi099 ceph-mon[31851]: pgmap v89: 97 pgs: 97 active+clean; 31 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 136 op/s 2024-05-14T17:06:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:31 smithi193 ceph-mon[35962]: pgmap v89: 97 pgs: 97 active+clean; 31 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 136 op/s 2024-05-14T17:06:31.446 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '380494ce', quiesce took 5.7 seconds. Will release after: 21.0 2024-05-14T17:06:31.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:31 smithi150 ceph-mon[36204]: pgmap v89: 97 pgs: 97 active+clean; 31 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 136 op/s 2024-05-14T17:06:31.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 5.06 MB/sec warmup 43 sec latency 833.402 ms 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 -- 172.21.15.99:0/2131618083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c101b70 msgr2=0x7fa14c1076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 --2- 172.21.15.99:0/2131618083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c101b70 0x7fa14c1076a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa134009900 tx=0x7fa13402a410 comp rx=0 tx=0).stop 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 -- 172.21.15.99:0/2131618083 shutdown_connections 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 --2- 172.21.15.99:0/2131618083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c107d80 0x7fa14c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 --2- 172.21.15.99:0/2131618083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c101b70 0x7fa14c1076a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 --2- 172.21.15.99:0/2131618083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c1011c0 0x7fa14c1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 -- 172.21.15.99:0/2131618083 >> 172.21.15.99:0/2131618083 conn(0x7fa14c077ee0 msgr2=0x7fa14c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 -- 172.21.15.99:0/2131618083 shutdown_connections 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 -- 172.21.15.99:0/2131618083 wait complete. 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.968+0000 7fa150b04640 1 Processor -- start 2024-05-14T17:06:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 -- start start 2024-05-14T17:06:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 0x7fa14c102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c107d80 0x7fa14c102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c102bb0 0x7fa14c1a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c112ce0 con 0x7fa14c1011c0 2024-05-14T17:06:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c112b60 con 0x7fa14c102bb0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa150b04640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c112e60 con 0x7fa14c107d80 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 0x7fa14c102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 0x7fa14c102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56780/0 (socket says 172.21.15.99:56780) 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 -- 172.21.15.99:0/242100492 learned_addr learned my addr 172.21.15.99:0/242100492 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c102bb0 0x7fa14c1a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa149d9b640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c107d80 0x7fa14c102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 -- 172.21.15.99:0/242100492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c107d80 msgr2=0x7fa14c102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c107d80 0x7fa14c102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 -- 172.21.15.99:0/242100492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c102bb0 msgr2=0x7fa14c1a73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c102bb0 0x7fa14c1a73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 -- 172.21.15.99:0/242100492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c1a7a90 con 0x7fa14c1011c0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa14a59c640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 0x7fa14c102130 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00d950 tx=0x7fa13c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa1437fe640 1 -- 172.21.15.99:0/242100492 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c017030 con 0x7fa14c1011c0 2024-05-14T17:06:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.969+0000 7fa1437fe640 1 -- 172.21.15.99:0/242100492 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c004020 con 0x7fa14c1011c0 2024-05-14T17:06:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.970+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c1a7d80 con 0x7fa14c1011c0 2024-05-14T17:06:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.970+0000 7fa1437fe640 1 -- 172.21.15.99:0/242100492 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0047a0 con 0x7fa14c1011c0 2024-05-14T17:06:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.970+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c1a8250 con 0x7fa14c1011c0 2024-05-14T17:06:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.972+0000 7fa1437fe640 1 -- 172.21.15.99:0/242100492 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa13c009a30 con 0x7fa14c1011c0 2024-05-14T17:06:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.972+0000 7fa15381c740 1 Processor -- start 2024-05-14T17:06:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.972+0000 7fa15381c740 1 -- start start 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.972+0000 7fa15381c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.972+0000 7fa15381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.972+0000 7fa15381c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa15381c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556dba350c30 con 0x556dba3dd8f0 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa15381c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556dba3407d0 con 0x556dba3e8ad0 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa15381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556dba344810 con 0x556dba3de210 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ddcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56796/0 (socket says 172.21.15.99:56796) 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 -- 172.21.15.99:0/252224497 learned_addr learned my addr 172.21.15.99:0/252224497 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14a59c640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa149d9b640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 -- 172.21.15.99:0/252224497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 msgr2=0x556dba3e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 -- 172.21.15.99:0/252224497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 msgr2=0x556dba3eaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3eaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 -- 172.21.15.99:0/252224497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dba055210 con 0x556dba3dd8f0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14a59c640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa149d9b640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa1437fe640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1380b2a90 0x7fa1380b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14a59c640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1380b2a90 0x7fa1380b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ddcd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa144009f40 tx=0x7fa144010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e1f94bcde6139 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa140ff9640 1 -- 172.21.15.99:0/252224497 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144003b70 con 0x556dba3dd8f0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa1437fe640 1 -- 172.21.15.99:0/242100492 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa13c0e2080 con 0x7fa14c1011c0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa140ff9640 1 -- 172.21.15.99:0/252224497 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa144003d10 con 0x556dba3dd8f0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.973+0000 7fa140ff9640 1 -- 172.21.15.99:0/252224497 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1440119c0 con 0x556dba3dd8f0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 -- 172.21.15.99:0/252224497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 msgr2=0x556dba3ddcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa14a59c640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1380b2a90 0x7fa1380b4f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0df8b0 tx=0x7fa13c0e5900 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ddcd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa144009f40 tx=0x7fa144010860 comp rx=0 tx=0).stop 2024-05-14T17:06:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 -- 172.21.15.99:0/252224497 shutdown_connections 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 --2- 172.21.15.99:0/252224497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ddcd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 -- 172.21.15.99:0/252224497 >> 172.21.15.99:0/252224497 conn(0x556dba3d9e00 msgr2=0x556dba3da1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 -- 172.21.15.99:0/252224497 shutdown_connections 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.974+0000 7fa15381c740 1 -- 172.21.15.99:0/252224497 wait complete. 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 -- start start 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 Processor -- start 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3dd8f0 0x556dba3e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 0x556dba3e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba3e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556dba026790 con 0x556dba3de210 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556dba2fc890 con 0x556dba3dd8f0 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556dba3eb7a0 con 0x556dba3e8ad0 2024-05-14T17:06:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa15381c740 10 client.0 _ll_register_callbacks cb 0x556dba2ffd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 0x556dba3e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 0x556dba3e5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56806/0 (socket says 172.21.15.99:56806) 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 -- 172.21.15.99:0/3265715532 learned_addr learned my addr 172.21.15.99:0/3265715532 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3dd8f0 0x556dba3e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba3e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 -- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 msgr2=0x556dba3e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba3e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 -- 172.21.15.99:0/3265715532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3dd8f0 msgr2=0x556dba3e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3dd8f0 0x556dba3e84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa149d9b640 1 -- 172.21.15.99:0/3265715532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dba121620 con 0x556dba3de210 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.975+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba3e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa149d9b640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 0x556dba3e5790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa134006010 tx=0x7fa134032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2740634e4b6e4a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa131ffb640 1 -- 172.21.15.99:0/3265715532 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13403f070 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa131ffb640 1 -- 172.21.15.99:0/3265715532 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa134004040 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa144003d10 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa131ffb640 1 -- 172.21.15.99:0/3265715532 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa13404b050 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 10 client.15669 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556dba3f7190 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa131ffb640 1 -- 172.21.15.99:0/3265715532 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa144003d10 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa131ffb640 1 -- 172.21.15.99:0/3265715532 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa1340046c0 con 0x556dba3de210 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 10 client.15669 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 20 client.15669 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba3f8650 0x556dba3faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 4 client.15669 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.976+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dba352b80 con 0x556dba3f8650 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.977+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba3f8650 0x556dba3faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.977+0000 7fa131ffb640 10 client.15669 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.977+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba3f8650 0x556dba3faa40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641c138d4e207cb0 server_cookie=9f748f8abd280e49 in_seq=0 out_seq=0 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa131ffb640 1 -- 172.21.15.99:0/3265715532 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556dba352b80 con 0x556dba3f8650 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa131ffb640 10 client.15669 handle_command_reply: tid=0 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 client.15669 shutdown 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 msgr2=0x556dba3e5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 0x556dba3e5790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa134006010 tx=0x7fa134032e20 comp rx=0 tx=0).stop 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 shutdown_connections 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba3f8650 0x556dba3faa40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba3e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3de210 0x556dba3e5790 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa131ffb640 0 client.15669 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 --2- 172.21.15.99:0/3265715532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3dd8f0 0x556dba3e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 >> 172.21.15.99:0/3265715532 conn(0x556dba3d9e00 msgr2=0x556dba3d9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 shutdown_connections 2024-05-14T17:06:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 1 -- 172.21.15.99:0/3265715532 wait complete. 2024-05-14T17:06:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.978+0000 7fa15381c740 20 client.15669 trim_cache size 0 max 16384 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 Processor -- start 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 -- start start 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3de210 0x556dba400310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba400850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1b30a0 con 0x556dba3dd8f0 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1b0970 con 0x556dba3de210 2024-05-14T17:06:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa15381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1ae7e0 con 0x556dba3e8ad0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56812/0 (socket says 172.21.15.99:56812) 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 -- 172.21.15.99:0/1427001143 learned_addr learned my addr 172.21.15.99:0/1427001143 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 -- 172.21.15.99:0/1427001143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 msgr2=0x556dba400850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba400850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 -- 172.21.15.99:0/1427001143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3de210 msgr2=0x556dba400310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3de210 0x556dba400310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.993+0000 7fa14ad9d640 1 -- 172.21.15.99:0/1427001143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dba3e4740 con 0x556dba3dd8f0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3f4670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa144011610 tx=0x7fa144009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48def30f26d440ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa133fff640 1 -- 172.21.15.99:0/1427001143 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144003c70 con 0x556dba3dd8f0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa133fff640 1 -- 172.21.15.99:0/1427001143 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa144003e10 con 0x556dba3dd8f0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa133fff640 1 -- 172.21.15.99:0/1427001143 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144005030 con 0x556dba3dd8f0 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 -- 172.21.15.99:0/1427001143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 msgr2=0x556dba3f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3f4670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa144011610 tx=0x7fa144009500 comp rx=0 tx=0).stop 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 -- 172.21.15.99:0/1427001143 shutdown_connections 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3e8ad0 0x556dba400850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3de210 0x556dba400310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 --2- 172.21.15.99:0/1427001143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3f4670 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 -- 172.21.15.99:0/1427001143 >> 172.21.15.99:0/1427001143 conn(0x556dba3d9e00 msgr2=0x556dba3d9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 -- 172.21.15.99:0/1427001143 shutdown_connections 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.994+0000 7fa15381c740 1 -- 172.21.15.99:0/1427001143 wait complete. 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 -- start start 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 Processor -- start 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1aee30 con 0x556dba3dd8f0 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1b02a0 con 0x556dba3e8ad0 2024-05-14T17:06:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1af500 con 0x556dba3de210 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa15381c740 10 client.0 _ll_register_callbacks cb 0x7fa134005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ee7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56814/0 (socket says 172.21.15.99:56814) 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 -- 172.21.15.99:0/348707801 learned_addr learned my addr 172.21.15.99:0/348707801 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa149d9b640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 -- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 msgr2=0x556dba3eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 -- 172.21.15.99:0/348707801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 msgr2=0x556dba3ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 -- 172.21.15.99:0/348707801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dba3fadb0 con 0x556dba3dd8f0 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa149d9b640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa14ad9d640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ee7f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa144009d50 tx=0x7fa144003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67850b71815215bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.995+0000 7fa131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa131ffb640 1 -- 172.21.15.99:0/348707801 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144011720 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa131ffb640 1 -- 172.21.15.99:0/348707801 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa1440118c0 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa144003e10 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa131ffb640 1 -- 172.21.15.99:0/348707801 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa144026470 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 10 client.15675 fetch_fsmap learned FSMap version 26 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556dba463240 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa131ffb640 1 -- 172.21.15.99:0/348707801 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa144003e10 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa131ffb640 1 -- 172.21.15.99:0/348707801 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa144005590 con 0x556dba3dd8f0 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 10 client.15675 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 20 client.15675 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba4636c0 0x556dba465ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 4 client.15675 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556dba34c360 con 0x556dba4636c0 2024-05-14T17:06:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.996+0000 7fa149d9b640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba4636c0 0x556dba465ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.997+0000 7fa131ffb640 10 client.15675 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:31.997+0000 7fa149d9b640 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba4636c0 0x556dba465ab0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36edf813a6be376 server_cookie=19b296e5d0d8718 in_seq=0 out_seq=0 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa131ffb640 1 -- 172.21.15.99:0/348707801 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556dba34c360 con 0x556dba4636c0 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa131ffb640 10 client.15675 handle_command_reply: tid=0 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 client.15675 shutdown 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 msgr2=0x556dba3ee7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ee7f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa144009d50 tx=0x7fa144003d30 comp rx=0 tx=0).stop 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 shutdown_connections 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556dba4636c0 0x556dba465ab0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556dba3e8ad0 0x556dba3ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dba3de210 0x556dba3eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa131ffb640 0 client.15675 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 --2- 172.21.15.99:0/348707801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556dba3dd8f0 0x556dba3ee7f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 >> 172.21.15.99:0/348707801 conn(0x556dba3d9e00 msgr2=0x556dba3f3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 shutdown_connections 2024-05-14T17:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 1 -- 172.21.15.99:0/348707801 wait complete. 2024-05-14T17:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa15381c740 20 client.15675 trim_cache size 0 max 16384 2024-05-14T17:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1380b2a90 msgr2=0x7fa1380b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa150b04640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1380b2a90 0x7fa1380b4f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0df8b0 tx=0x7fa13c0e5900 comp rx=0 tx=0).stop 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.064+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 msgr2=0x7fa14c102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 0x7fa14c102130 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00d950 tx=0x7fa13c00de10 comp rx=0 tx=0).stop 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 shutdown_connections 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1380b2a90 0x7fa1380b4f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c102bb0 0x7fa14c1a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c107d80 0x7fa14c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 --2- 172.21.15.99:0/242100492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa14c1011c0 0x7fa14c102130 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 >> 172.21.15.99:0/242100492 conn(0x7fa14c077ee0 msgr2=0x7fa14c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 shutdown_connections 2024-05-14T17:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:32.065+0000 7fa150b04640 1 -- 172.21.15.99:0/242100492 wait complete. 2024-05-14T17:06:32.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.94 MB/sec warmup 44 sec latency 1833.472 ms 2024-05-14T17:06:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:33 smithi099 ceph-mon[31851]: pgmap v90: 97 pgs: 97 active+clean; 31 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 597 KiB/s wr, 79 op/s 2024-05-14T17:06:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:33 smithi193 ceph-mon[35962]: pgmap v90: 97 pgs: 97 active+clean; 31 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 597 KiB/s wr, 79 op/s 2024-05-14T17:06:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:33 smithi150 ceph-mon[36204]: pgmap v90: 97 pgs: 97 active+clean; 31 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 597 KiB/s wr, 79 op/s 2024-05-14T17:06:33.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.83 MB/sec warmup 45 sec latency 2833.537 ms 2024-05-14T17:06:34.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.73 MB/sec warmup 46 sec latency 3833.625 ms 2024-05-14T17:06:35.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:35 smithi099 ceph-mon[31851]: pgmap v91: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 951 KiB/s wr, 166 op/s 2024-05-14T17:06:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:35 smithi193 ceph-mon[35962]: pgmap v91: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 951 KiB/s wr, 166 op/s 2024-05-14T17:06:35.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:35 smithi150 ceph-mon[36204]: pgmap v91: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 951 KiB/s wr, 166 op/s 2024-05-14T17:06:35.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.63 MB/sec warmup 47 sec latency 4833.730 ms 2024-05-14T17:06:36.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:36 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:36.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:36 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:36.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:36 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:36.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.53 MB/sec warmup 48 sec latency 5833.781 ms 2024-05-14T17:06:37.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:37 smithi193 ceph-mon[35962]: pgmap v92: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 156 op/s 2024-05-14T17:06:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:37 smithi150 ceph-mon[36204]: pgmap v92: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 156 op/s 2024-05-14T17:06:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:37 smithi099 ceph-mon[31851]: pgmap v92: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 156 op/s 2024-05-14T17:06:37.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.44 MB/sec warmup 49 sec latency 6833.837 ms 2024-05-14T17:06:38.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.35 MB/sec warmup 50 sec latency 7833.882 ms 2024-05-14T17:06:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:39 smithi193 ceph-mon[35962]: pgmap v93: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 156 op/s 2024-05-14T17:06:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:39 smithi150 ceph-mon[36204]: pgmap v93: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 156 op/s 2024-05-14T17:06:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:39 smithi099 ceph-mon[31851]: pgmap v93: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 156 op/s 2024-05-14T17:06:39.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.26 MB/sec warmup 51 sec latency 8833.930 ms 2024-05-14T17:06:40.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.18 MB/sec warmup 52 sec latency 9834.025 ms 2024-05-14T17:06:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:41 smithi193 ceph-mon[35962]: pgmap v94: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 159 op/s 2024-05-14T17:06:41.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:41 smithi150 ceph-mon[36204]: pgmap v94: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 159 op/s 2024-05-14T17:06:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:41 smithi099 ceph-mon[31851]: pgmap v94: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 927 KiB/s wr, 159 op/s 2024-05-14T17:06:41.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.10 MB/sec warmup 53 sec latency 10834.140 ms 2024-05-14T17:06:42.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 4.03 MB/sec warmup 54 sec latency 11834.267 ms 2024-05-14T17:06:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:43 smithi193 ceph-mon[35962]: pgmap v95: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 354 KiB/s wr, 89 op/s 2024-05-14T17:06:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:43 smithi150 ceph-mon[36204]: pgmap v95: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 354 KiB/s wr, 89 op/s 2024-05-14T17:06:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:43 smithi099 ceph-mon[31851]: pgmap v95: 97 pgs: 97 active+clean; 26 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 354 KiB/s wr, 89 op/s 2024-05-14T17:06:43.519 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.95 MB/sec warmup 55 sec latency 12834.303 ms 2024-05-14T17:06:44.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.88 MB/sec warmup 56 sec latency 13834.387 ms 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 -- 172.21.15.99:0/3864037443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc107a70 msgr2=0x7fe4cc107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 --2- 172.21.15.99:0/3864037443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc107a70 0x7fe4cc107e50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009930 tx=0x7fe4c802a420 comp rx=0 tx=0).stop 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 -- 172.21.15.99:0/3864037443 shutdown_connections 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 --2- 172.21.15.99:0/3864037443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 --2- 172.21.15.99:0/3864037443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc101c30 0x7fe4cc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 --2- 172.21.15.99:0/3864037443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc107a70 0x7fe4cc107e50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 -- 172.21.15.99:0/3864037443 >> 172.21.15.99:0/3864037443 conn(0x7fe4cc0786a0 msgr2=0x7fe4cc078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 -- 172.21.15.99:0/3864037443 shutdown_connections 2024-05-14T17:06:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.162+0000 7fe4d335f640 1 -- 172.21.15.99:0/3864037443 wait complete. 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 Processor -- start 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 -- start start 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4cc101c30 0x7fe4cc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc107a70 0x7fe4cc19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc111bf0 con 0x7fe4cc1025f0 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc111a70 con 0x7fe4cc101c30 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d335f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc111d70 con 0x7fe4cc107a70 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d08fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d08fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52130/0 (socket says 172.21.15.99:52130) 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2151783309 learned_addr learned my addr 172.21.15.99:0/2151783309 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.163+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4cc101c30 0x7fe4cc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc107a70 0x7fe4cc19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2151783309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc107a70 msgr2=0x7fe4cc19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc107a70 0x7fe4cc19da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2151783309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4cc101c30 msgr2=0x7fe4cc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4cc101c30 0x7fe4cc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2151783309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc19e1c0 con 0x7fe4cc1025f0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc199710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b4a0 tx=0x7fe4bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4c27fc640 1 -- 172.21.15.99:0/2151783309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc014070 con 0x7fe4cc1025f0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4c27fc640 1 -- 172.21.15.99:0/2151783309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4bc010ac0 con 0x7fe4cc1025f0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4c27fc640 1 -- 172.21.15.99:0/2151783309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc017c30 con 0x7fe4cc1025f0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc19e4b0 con 0x7fe4cc1025f0 2024-05-14T17:06:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.164+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a5d40 con 0x7fe4cc1025f0 2024-05-14T17:06:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4c27fc640 1 -- 172.21.15.99:0/2151783309 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe4bc017070 con 0x7fe4cc1025f0 2024-05-14T17:06:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 Processor -- start 2024-05-14T17:06:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 -- start start 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd88dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88e210 0x5587bd898590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd89aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587bd800c30 con 0x5587bd898ad0 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587bd7f07d0 con 0x5587bd88e210 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d6078740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587bd7f4810 con 0x5587bd88d8f0 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d08fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd89aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d08fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd89aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52134/0 (socket says 172.21.15.99:52134) 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d08fa640 1 -- 172.21.15.99:0/1376347049 learned_addr learned my addr 172.21.15.99:0/1376347049 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.167+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88e210 0x5587bd898590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd88dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d08fa640 1 -- 172.21.15.99:0/1376347049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 msgr2=0x5587bd88dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd88dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d08fa640 1 -- 172.21.15.99:0/1376347049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88e210 msgr2=0x5587bd898590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88e210 0x5587bd898590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d08fa640 1 -- 172.21.15.99:0/1376347049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bd505210 con 0x5587bd898ad0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4c27fc640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4b80b2a10 0x7fe4b80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4b80b2a10 0x7fe4b80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd89aec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0df080 tx=0x7fe4bc0e9690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97fee4835ed17fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4c27fc640 1 -- 172.21.15.99:0/2151783309 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe4bc0e1880 con 0x7fe4cc1025f0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4b77fe640 1 -- 172.21.15.99:0/1376347049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0a4b60 con 0x5587bd898ad0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4b77fe640 1 -- 172.21.15.99:0/1376347049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4bc0a4420 con 0x5587bd898ad0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.168+0000 7fe4b77fe640 1 -- 172.21.15.99:0/1376347049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0a8410 con 0x5587bd898ad0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 -- 172.21.15.99:0/1376347049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 msgr2=0x5587bd89aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4b80b2a10 0x7fe4b80b4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4008e40 tx=0x7fe4c40085f0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd89aec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0df080 tx=0x7fe4bc0e9690 comp rx=0 tx=0).stop 2024-05-14T17:06:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 -- 172.21.15.99:0/1376347049 shutdown_connections 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd89aec0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88e210 0x5587bd898590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1376347049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd88dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 -- 172.21.15.99:0/1376347049 >> 172.21.15.99:0/1376347049 conn(0x5587bd889e00 msgr2=0x5587bd88a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 -- 172.21.15.99:0/1376347049 shutdown_connections 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.169+0000 7fe4d6078740 1 -- 172.21.15.99:0/1376347049 wait complete. 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 -- start start 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 Processor -- start 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd898420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 0x5587bd895710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd895c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587bd4d6790 con 0x5587bd88e210 2024-05-14T17:06:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587bd7ac890 con 0x5587bd898ad0 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587bd89b7a0 con 0x5587bd88d8f0 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d6078740 10 client.0 _ll_register_callbacks cb 0x5587bd7afd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d08fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 0x5587bd895710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d08fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 0x5587bd895710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52142/0 (socket says 172.21.15.99:52142) 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2200339553 learned_addr learned my addr 172.21.15.99:0/2200339553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd895c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.170+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd898420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 msgr2=0x5587bd898420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd898420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2200339553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 msgr2=0x5587bd895c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd895c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d08fa640 1 -- 172.21.15.99:0/2200339553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bd5d1620 con 0x5587bd88e210 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd895c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd898420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 0x5587bd895710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004700 tx=0x7fe4bc0e73a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc84b0c704f93f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/2200339553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc01e070 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/2200339553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe4bc0df280 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc0a4420 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/2200339553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4bc0a85f0 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587bd8a71c0 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/2200339553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4bc0a4420 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/2200339553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe4bc0a4f30 con 0x5587bd88e210 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 20 client.15687 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd8a8510 0x5587bd8aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 4 client.15687 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.171+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587bd802b80 con 0x5587bd8a8510 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.172+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd8a8510 0x5587bd8aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.172+0000 7fe4b4ff9640 10 client.15687 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.172+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd8a8510 0x5587bd8aa900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0abe35187063310 server_cookie=846a954ed138a78f in_seq=0 out_seq=0 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/2200339553 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587bd802b80 con 0x5587bd8a8510 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4b4ff9640 10 client.15687 handle_command_reply: tid=0 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 client.15687 shutdown 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 msgr2=0x5587bd895710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 0x5587bd895710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004700 tx=0x7fe4bc0e73a0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 shutdown_connections 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd8a8510 0x5587bd8aa900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd895c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88e210 0x5587bd895710 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 --2- 172.21.15.99:0/2200339553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88d8f0 0x5587bd898420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4b4ff9640 0 client.15687 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 >> 172.21.15.99:0/2200339553 conn(0x5587bd889e00 msgr2=0x5587bd889100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 shutdown_connections 2024-05-14T17:06:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.173+0000 7fe4d6078740 1 -- 172.21.15.99:0/2200339553 wait complete. 2024-05-14T17:06:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.174+0000 7fe4d6078740 20 client.15687 trim_cache size 0 max 16384 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 Processor -- start 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 -- start start 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88d8f0 0x5587bd897f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd8ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1b0db0 con 0x5587bd898ad0 2024-05-14T17:06:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1ae680 con 0x5587bd88d8f0 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d6078740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1ac4d0 con 0x5587bd88e210 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd8ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd8ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52150/0 (socket says 172.21.15.99:52150) 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 -- 172.21.15.99:0/3326367481 learned_addr learned my addr 172.21.15.99:0/3326367481 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88d8f0 0x5587bd897f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 -- 172.21.15.99:0/3326367481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 msgr2=0x5587bd8ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 -- 172.21.15.99:0/3326367481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88d8f0 msgr2=0x5587bd897f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88d8f0 0x5587bd897f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 -- 172.21.15.99:0/3326367481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bd889c30 con 0x5587bd898ad0 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88d8f0 0x5587bd897f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.189+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd8ade60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4008c20 tx=0x7fe4c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8c2d7019c67881 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4b6ffd640 1 -- 172.21.15.99:0/3326367481 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c402b070 con 0x5587bd898ad0 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4b6ffd640 1 -- 172.21.15.99:0/3326367481 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c401ccf0 con 0x5587bd898ad0 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4b6ffd640 1 -- 172.21.15.99:0/3326367481 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4026e40 con 0x5587bd898ad0 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 -- 172.21.15.99:0/3326367481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 msgr2=0x5587bd8ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd8ade60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4008c20 tx=0x7fe4c4014000 comp rx=0 tx=0).stop 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 -- 172.21.15.99:0/3326367481 shutdown_connections 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd898ad0 0x5587bd8ade60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 --2- 172.21.15.99:0/3326367481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd88d8f0 0x5587bd897f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 -- 172.21.15.99:0/3326367481 >> 172.21.15.99:0/3326367481 conn(0x5587bd889e00 msgr2=0x5587bd88fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 -- 172.21.15.99:0/3326367481 shutdown_connections 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.190+0000 7fe4d6078740 1 -- 172.21.15.99:0/3326367481 wait complete. 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 -- start start 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 Processor -- start 2024-05-14T17:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 0x5587bd8adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd8ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1acb40 con 0x5587bd88d8f0 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1adfb0 con 0x5587bd898ad0 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1ad210 con 0x5587bd88e210 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d6078740 10 client.0 _ll_register_callbacks cb 0x7fe4c4008490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 0x5587bd8adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 0x5587bd8adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52154/0 (socket says 172.21.15.99:52154) 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 -- 172.21.15.99:0/1902683909 learned_addr learned my addr 172.21.15.99:0/1902683909 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d18fc640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd8ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 -- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 msgr2=0x5587bd8aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 -- 172.21.15.99:0/1902683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 msgr2=0x5587bd8ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd8ab530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.191+0000 7fe4d10fb640 1 -- 172.21.15.99:0/1902683909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bd8abaa0 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d10fb640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 0x5587bd8adc10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8006010 tx=0x7fe4c80316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbec259aa9b5ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/1902683909 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c804b020 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/1902683909 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe4c8004240 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c401ccf0 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/1902683909 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4c8056050 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5587bd912790 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/1902683909 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c401ccf0 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/1902683909 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe4c8003780 con 0x5587bd88d8f0 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 20 client.15693 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd912f70 0x5587bd915390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 4 client.15693 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.192+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587bd7fc360 con 0x5587bd912f70 2024-05-14T17:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.193+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd912f70 0x5587bd915390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.193+0000 7fe4b4ff9640 10 client.15693 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.193+0000 7fe4d08fa640 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd912f70 0x5587bd915390 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173f9c814579db6f server_cookie=7c579dadc7056616 in_seq=0 out_seq=0 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4b4ff9640 1 -- 172.21.15.99:0/1902683909 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5587bd7fc360 con 0x5587bd912f70 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4b4ff9640 10 client.15693 handle_command_reply: tid=0 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 client.15693 shutdown 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 msgr2=0x5587bd8adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 0x5587bd8adc10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8006010 tx=0x7fe4c80316a0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 shutdown_connections 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5587bd912f70 0x5587bd915390 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587bd898ad0 0x5587bd8ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587bd88e210 0x5587bd8aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4b4ff9640 0 client.15693 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 --2- 172.21.15.99:0/1902683909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587bd88d8f0 0x5587bd8adc10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 >> 172.21.15.99:0/1902683909 conn(0x5587bd889e00 msgr2=0x5587bd8a39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 shutdown_connections 2024-05-14T17:06:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 1 -- 172.21.15.99:0/1902683909 wait complete. 2024-05-14T17:06:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.212+0000 7fe4d6078740 20 client.15693 trim_cache size 0 max 16384 2024-05-14T17:06:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:06:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:06:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4b80b2a10 msgr2=0x7fe4b80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4b80b2a10 0x7fe4b80b4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4008e40 tx=0x7fe4c40085f0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 msgr2=0x7fe4cc199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc199710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b4a0 tx=0x7fe4bc00b960 comp rx=0 tx=0).stop 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 shutdown_connections 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe4b80b2a10 0x7fe4b80b4ed0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc107a70 0x7fe4cc19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc1025f0 0x7fe4cc199710 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 --2- 172.21.15.99:0/2151783309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4cc101c30 0x7fe4cc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 >> 172.21.15.99:0/2151783309 conn(0x7fe4cc0786a0 msgr2=0x7fe4cc06b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 shutdown_connections 2024-05-14T17:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:45.213+0000 7fe4d335f640 1 -- 172.21.15.99:0/2151783309 wait complete. 2024-05-14T17:06:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:45 smithi193 ceph-mon[35962]: pgmap v96: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 434 KiB/s wr, 93 op/s 2024-05-14T17:06:45.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:45 smithi150 ceph-mon[36204]: pgmap v96: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 434 KiB/s wr, 93 op/s 2024-05-14T17:06:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:45 smithi099 ceph-mon[31851]: pgmap v96: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 434 KiB/s wr, 93 op/s 2024-05-14T17:06:45.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.81 MB/sec warmup 57 sec latency 14834.528 ms 2024-05-14T17:06:46.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.75 MB/sec warmup 58 sec latency 15834.621 ms 2024-05-14T17:06:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:47 smithi193 ceph-mon[35962]: pgmap v97: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 79 KiB/s wr, 6 op/s 2024-05-14T17:06:47.316 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:47 smithi099 ceph-mon[31851]: pgmap v97: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 79 KiB/s wr, 6 op/s 2024-05-14T17:06:47.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:47 smithi150 ceph-mon[36204]: pgmap v97: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 79 KiB/s wr, 6 op/s 2024-05-14T17:06:47.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.69 MB/sec warmup 59 sec latency 16834.689 ms 2024-05-14T17:06:48.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.62 MB/sec warmup 60 sec latency 17834.740 ms 2024-05-14T17:06:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:49 smithi193 ceph-mon[35962]: pgmap v98: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 79 KiB/s wr, 6 op/s 2024-05-14T17:06:49.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:49 smithi150 ceph-mon[36204]: pgmap v98: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 79 KiB/s wr, 6 op/s 2024-05-14T17:06:49.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:49 smithi099 ceph-mon[31851]: pgmap v98: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 79 KiB/s wr, 6 op/s 2024-05-14T17:06:49.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.56 MB/sec warmup 61 sec latency 18834.782 ms 2024-05-14T17:06:50.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.51 MB/sec warmup 62 sec latency 19834.834 ms 2024-05-14T17:06:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:51 smithi193 ceph-mon[35962]: pgmap v99: 97 pgs: 97 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 98 KiB/s wr, 14 op/s 2024-05-14T17:06:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:51 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:51.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:51 smithi150 ceph-mon[36204]: pgmap v99: 97 pgs: 97 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 98 KiB/s wr, 14 op/s 2024-05-14T17:06:51.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:51 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:51 smithi099 ceph-mon[31851]: pgmap v99: 97 pgs: 97 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 98 KiB/s wr, 14 op/s 2024-05-14T17:06:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:51 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:51.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.45 MB/sec warmup 63 sec latency 20834.862 ms 2024-05-14T17:06:52.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:06:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:06:52.475 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:52.475 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:52.475 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:06:52.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.40 MB/sec warmup 64 sec latency 21834.890 ms 2024-05-14T17:06:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:06:52.701 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:53.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:53 smithi150 ceph-mon[36204]: pgmap v100: 97 pgs: 97 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 98 KiB/s wr, 11 op/s 2024-05-14T17:06:53.499 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:53 smithi099 ceph-mon[31851]: pgmap v100: 97 pgs: 97 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 98 KiB/s wr, 11 op/s 2024-05-14T17:06:53.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.35 MB/sec warmup 65 sec latency 22834.917 ms 2024-05-14T17:06:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:53 smithi193 ceph-mon[35962]: pgmap v100: 97 pgs: 97 active+clean; 27 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 98 KiB/s wr, 11 op/s 2024-05-14T17:06:54.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 -- 172.21.15.99:0/230578762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20103850 msgr2=0x7f2b20103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 --2- 172.21.15.99:0/230578762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20103850 0x7f2b20103c30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2b100099c0 tx=0x7f2b10009cd0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 -- 172.21.15.99:0/230578762 shutdown_connections 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 --2- 172.21.15.99:0/230578762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b20108920 0x7f2b201104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 --2- 172.21.15.99:0/230578762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20103850 0x7f2b20103c30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 --2- 172.21.15.99:0/230578762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b20104200 0x7f2b201081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 -- 172.21.15.99:0/230578762 >> 172.21.15.99:0/230578762 conn(0x7f2b2007a620 msgr2=0x7f2b2007aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 -- 172.21.15.99:0/230578762 shutdown_connections 2024-05-14T17:06:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.458+0000 7f2b262f0700 1 -- 172.21.15.99:0/230578762 wait complete. 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 Processor -- start 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 -- start start 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b20103850 0x7f2b20075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 0x7f2b20075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b20108920 0x7f2b200762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20113730 con 0x7f2b20104200 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201135b0 con 0x7f2b20108920 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b262f0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201138b0 con 0x7f2b20103850 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 0x7f2b20075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 0x7f2b20075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52176/0 (socket says 172.21.15.99:52176) 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 -- 172.21.15.99:0/1241066760 learned_addr learned my addr 172.21.15.99:0/1241066760 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1ffff700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b20103850 0x7f2b20075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b2488d700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b20108920 0x7f2b200762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 -- 172.21.15.99:0/1241066760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b20103850 msgr2=0x7f2b20075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b20103850 0x7f2b20075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 -- 172.21.15.99:0/1241066760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b20108920 msgr2=0x7f2b200762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b20108920 0x7f2b200762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.459+0000 7f2b1f7fe700 1 -- 172.21.15.99:0/1241066760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b201a8140 con 0x7f2b20104200 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.460+0000 7f2b1f7fe700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 0x7f2b20075d80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d980 tx=0x7f2b1400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.460+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14017070 con 0x7f2b20104200 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.460+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b14004830 con 0x7f2b20104200 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.460+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b201a8430 con 0x7f2b20104200 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.460+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005440 con 0x7f2b20104200 2024-05-14T17:06:54.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.460+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201a8920 con 0x7f2b20104200 2024-05-14T17:06:54.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.461+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2010bb60 con 0x7f2b20104200 2024-05-14T17:06:54.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.462+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2b140055e0 con 0x7f2b20104200 2024-05-14T17:06:54.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.463+0000 7f2b1d7fa700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2b080b29f0 0x7f2b080b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:54.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.463+0000 7f2b1ffff700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2b080b29f0 0x7f2b080b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:54.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.463+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2b140e29b0 con 0x7f2b20104200 2024-05-14T17:06:54.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.464+0000 7f2b1ffff700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2b080b29f0 0x7f2b080b4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b10003800 tx=0x7f2b1001a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.467+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b14020020 con 0x7f2b20104200 2024-05-14T17:06:54.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.29 MB/sec warmup 66 sec latency 23834.943 ms 2024-05-14T17:06:54.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.722+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b20076d20 con 0x7f2b20104200 2024-05-14T17:06:54.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.722+0000 7f2b1d7fa700 1 -- 172.21.15.99:0/1241066760 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11876 (secure 0 0 0) 0x7f2b140aa3c0 con 0x7f2b20104200 2024-05-14T17:06:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:06:54.722 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:06:34.984717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:06:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.724+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2b080b29f0 msgr2=0x7f2b080b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.724+0000 7f2b262f0700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2b080b29f0 0x7f2b080b4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b10003800 tx=0x7f2b1001a040 comp rx=0 tx=0).stop 2024-05-14T17:06:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.724+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 msgr2=0x7f2b20075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.724+0000 7f2b262f0700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 0x7f2b20075d80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d980 tx=0x7f2b1400dd40 comp rx=0 tx=0).stop 2024-05-14T17:06:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 shutdown_connections 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2b080b29f0 0x7f2b080b4eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b20108920 0x7f2b200762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b20103850 0x7f2b20075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 --2- 172.21.15.99:0/1241066760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b20104200 0x7f2b20075d80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 >> 172.21.15.99:0/1241066760 conn(0x7f2b2007a620 msgr2=0x7f2b20101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 shutdown_connections 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:54.725+0000 7f2b262f0700 1 -- 172.21.15.99:0/1241066760 wait complete. 2024-05-14T17:06:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-05-14T17:06:55.326 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:55 smithi099 ceph-mon[31851]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-05-14T17:06:55.326 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:55 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1241066760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:55.367 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 380494ce --release --await' 2024-05-14T17:06:55.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 380494ce --release --await 2024-05-14T17:06:55.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:55 smithi150 ceph-mon[36204]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-05-14T17:06:55.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:55 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1241066760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:55.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.25 MB/sec warmup 67 sec latency 24835.002 ms 2024-05-14T17:06:55.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:55 smithi193 ceph-mon[35962]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.0 MiB/s wr, 33 op/s 2024-05-14T17:06:55.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:55 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1241066760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:55.647 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:56.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37157 3.20 MB/sec warmup 68 sec latency 25835.081 ms 2024-05-14T17:06:57.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:57 smithi099 ceph-mon[31851]: pgmap v102: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 966 KiB/s wr, 29 op/s 2024-05-14T17:06:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 -- 172.21.15.99:0/3157677390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8102430 msgr2=0x7fe6d81028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/3157677390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8102430 0x7fe6d81028b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009990 tx=0x7fe6c8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 -- 172.21.15.99:0/3157677390 shutdown_connections 2024-05-14T17:06:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/3157677390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6d8102df0 0x7fe6d810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/3157677390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d8108510 0x7fe6d81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/3157677390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8102430 0x7fe6d81028b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 -- 172.21.15.99:0/3157677390 >> 172.21.15.99:0/3157677390 conn(0x7fe6d80793b0 msgr2=0x7fe6d80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 -- 172.21.15.99:0/3157677390 shutdown_connections 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.332+0000 7fe6dff2c700 1 -- 172.21.15.99:0/3157677390 wait complete. 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 Processor -- start 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 -- start start 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d8102430 0x7fe6d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6d8102df0 0x7fe6d819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 0x7fe6d81a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8112640 con 0x7fe6d8108510 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81124c0 con 0x7fe6d8102430 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dff2c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81127c0 con 0x7fe6d8102df0 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 0x7fe6d81a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 0x7fe6d81a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57082/0 (socket says 172.21.15.99:57082) 2024-05-14T17:06:57.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 -- 172.21.15.99:0/220908148 learned_addr learned my addr 172.21.15.99:0/220908148 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6dd4c7700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6d8102df0 0x7fe6d819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 -- 172.21.15.99:0/220908148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6d8102df0 msgr2=0x7fe6d819b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6d8102df0 0x7fe6d819b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 -- 172.21.15.99:0/220908148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d8102430 msgr2=0x7fe6d819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d8102430 0x7fe6d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d8102430 0x7fe6d819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 -- 172.21.15.99:0/220908148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81a60e0 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 0x7fe6d81a5ba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000ada0 tx=0x7fe6d000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.333+0000 7fe6ceffd700 1 -- 172.21.15.99:0/220908148 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0017020 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.334+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a6370 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.334+0000 7fe6ceffd700 1 -- 172.21.15.99:0/220908148 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d000ce30 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.334+0000 7fe6ceffd700 1 -- 172.21.15.99:0/220908148 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00054a0 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.334+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a6830 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.336+0000 7fe6ceffd700 1 -- 172.21.15.99:0/220908148 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe6d000c980 con 0x7fe6d8108510 2024-05-14T17:06:57.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.337+0000 7fe6ceffd700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe6c40b29f0 0x7fe6c40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.337+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe6c40b29f0 0x7fe6c40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.337+0000 7fe6ea7c2b80 1 Processor -- start 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.337+0000 7fe6ceffd700 1 -- 172.21.15.99:0/220908148 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe6d00e1e70 con 0x7fe6d8108510 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 -- start start 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe6c40b29f0 0x7fe6c40b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4006fd0 tx=0x7fe6d4008040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3baaa0 0x56261e3bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3bb3c0 0x56261e3bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56261e314d90 con 0x56261e3bf2d0 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56261e304930 con 0x56261e3baaa0 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56261e308970 con 0x56261e3bb3c0 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3bf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57096/0 (socket says 172.21.15.99:57096) 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 -- 172.21.15.99:0/118234268 learned_addr learned my addr 172.21.15.99:0/118234268 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6dd4c7700 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3bb3c0 0x56261e3bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3baaa0 0x56261e3bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 -- 172.21.15.99:0/118234268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3bb3c0 msgr2=0x56261e3bed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3bb3c0 0x56261e3bed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 -- 172.21.15.99:0/118234268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3baaa0 msgr2=0x56261e3bae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3baaa0 0x56261e3bae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6de4c9700 1 -- 172.21.15.99:0/118234268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261e2549d0 con 0x56261e3bf2d0 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.338+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3baaa0 0x56261e3bae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.339+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3bf6f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00a9140 tx=0x7fe6d00a8c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fd9dc0f90d19d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.339+0000 7fe6c3fff700 1 -- 172.21.15.99:0/118234268 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00a98b0 con 0x56261e3bf2d0 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.339+0000 7fe6c3fff700 1 -- 172.21.15.99:0/118234268 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d00a91b0 con 0x56261e3bf2d0 2024-05-14T17:06:57.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.339+0000 7fe6c3fff700 1 -- 172.21.15.99:0/118234268 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00ad120 con 0x56261e3bf2d0 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/118234268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 msgr2=0x56261e3bf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3bf6f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00a9140 tx=0x7fe6d00a8c30 comp rx=0 tx=0).stop 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/118234268 shutdown_connections 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3bf6f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3baaa0 0x56261e3bae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/118234268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3bb3c0 0x56261e3bed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/118234268 >> 172.21.15.99:0/118234268 conn(0x56261e2e1350 msgr2=0x56261e3b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/118234268 shutdown_connections 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/118234268 wait complete. 2024-05-14T17:06:57.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 -- start start 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.340+0000 7fe6ea7c2b80 1 Processor -- start 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3c6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56261e20b0a0 con 0x56261e3bf2d0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56261e2a68c0 con 0x56261e3bb3c0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56261e3c7500 con 0x56261e3baaa0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ea7c2b80 10 client.0 _ll_register_callbacks cb 0x56261e29b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57106/0 (socket says 172.21.15.99:57106) 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1407510741 learned_addr learned my addr 172.21.15.99:0/1407510741 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6dd4c7700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 msgr2=0x56261e3c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1407510741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 msgr2=0x56261e3c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1407510741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261e2a3790 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.341+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3c4020 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000a0a0 tx=0x7fe6d00df530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb332b0b6bd4b4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1407510741 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00c2070 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1407510741 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe6d00ad120 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d00a91b0 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1407510741 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6d00a98b0 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 10 client.15711 fetch_fsmap learned FSMap version 27 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56261e3d1bc0 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1407510741 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d00a91b0 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1407510741 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe6d00dfce0 con 0x56261e3bf2d0 2024-05-14T17:06:57.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 10 client.15711 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 20 client.15711 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e3d4090 0x56261e3d6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 4 client.15711 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.342+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56261e382460 con 0x56261e3d4090 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.343+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e3d4090 0x56261e3d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.343+0000 7fe6c17fa700 10 client.15711 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.343+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e3d4090 0x56261e3d6480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3e77a37f49aea6 server_cookie=4b56e40b3dd0e00e in_seq=0 out_seq=0 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1407510741 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56261e382460 con 0x56261e3d4090 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6c17fa700 10 client.15711 handle_command_reply: tid=0 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 client.15711 shutdown 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 msgr2=0x56261e3c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3c4020 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000a0a0 tx=0x7fe6d00df530 comp rx=0 tx=0).stop 2024-05-14T17:06:57.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 shutdown_connections 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e3d4090 0x56261e3d6480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3c4020 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6c17fa700 0 client.15711 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1407510741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 >> 172.21.15.99:0/1407510741 conn(0x56261e2e1350 msgr2=0x56261e3ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 shutdown_connections 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1407510741 wait complete. 2024-05-14T17:06:57.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.344+0000 7fe6ea7c2b80 20 client.15711 trim_cache size 0 max 16384 2024-05-14T17:06:57.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 Processor -- start 2024-05-14T17:06:57.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 -- start start 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ae570 con 0x56261e3bf2d0 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81abe00 con 0x56261e3bb3c0 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81a9c70 con 0x56261e3baaa0 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3becf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57120/0 (socket says 172.21.15.99:57120) 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.364+0000 7fe6de4c9700 1 -- 172.21.15.99:0/3247475270 learned_addr learned my addr 172.21.15.99:0/3247475270 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6dd4c7700 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6de4c9700 1 -- 172.21.15.99:0/3247475270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 msgr2=0x56261e3ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6de4c9700 1 -- 172.21.15.99:0/3247475270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 msgr2=0x56261e3be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3be7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6de4c9700 1 -- 172.21.15.99:0/3247475270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261e3c2050 con 0x56261e3bf2d0 2024-05-14T17:06:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3becf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00a5230 tx=0x7fe6d00e1d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f86c503f3c75754 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6c37fe700 1 -- 172.21.15.99:0/3247475270 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00c2070 con 0x56261e3bf2d0 2024-05-14T17:06:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6c37fe700 1 -- 172.21.15.99:0/3247475270 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d000cc70 con 0x56261e3bf2d0 2024-05-14T17:06:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.365+0000 7fe6c37fe700 1 -- 172.21.15.99:0/3247475270 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00a9410 con 0x56261e3bf2d0 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/3247475270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 msgr2=0x56261e3becf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3becf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00a5230 tx=0x7fe6d00e1d80 comp rx=0 tx=0).stop 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/3247475270 shutdown_connections 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3becf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/3247475270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/3247475270 >> 172.21.15.99:0/3247475270 conn(0x56261e2e1350 msgr2=0x56261e3be0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/3247475270 shutdown_connections 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/3247475270 wait complete. 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 -- start start 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 Processor -- start 2024-05-14T17:06:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3d9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.366+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81aa2c0 con 0x56261e3bf2d0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ab730 con 0x56261e3bb3c0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81aa990 con 0x56261e3baaa0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 10 client.0 _ll_register_callbacks cb 0x56261e29b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ddcc8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3d9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3d86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57136/0 (socket says 172.21.15.99:57136) 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1377586400 learned_addr learned my addr 172.21.15.99:0/1377586400 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6dd4c7700 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 msgr2=0x56261e3d9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3d9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1377586400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 msgr2=0x56261e3d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 -- 172.21.15.99:0/1377586400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261e43e5c0 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6de4c9700 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3d86a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0068e50 tx=0x7fe6d00ad490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9ec600c53b948d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1377586400 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00c2070 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1377586400 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe6d00a9620 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56261e316ce0 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1377586400 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6d00a91b0 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 10 client.15717 fetch_fsmap learned FSMap version 27 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.367+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56261e43e5c0 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1377586400 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56261e316ce0 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1377586400 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe6d00dfce0 con 0x56261e3bf2d0 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ea7c2b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ea7c2b80 10 client.15717 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ea7c2b80 20 client.15717 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e43fb20 0x56261e441f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ea7c2b80 4 client.15717 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "380494ce", "release": true, "await": true}] 2024-05-14T17:06:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "380494ce", "release": true, "await": true}) -- 0x56261e3104c0 con 0x56261e43fb20 2024-05-14T17:06:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e43fb20 0x56261e441f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6c17fa700 10 client.15717 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.368+0000 7fe6ddcc8700 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e43fb20 0x56261e441f40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f79e1bcd7c15659 server_cookie=5c1852029100a2ef in_seq=0 out_seq=0 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6c17fa700 1 -- 172.21.15.99:0/1377586400 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56261e3104c0 con 0x56261e43fb20 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6c17fa700 10 client.15717 handle_command_reply: tid=0 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 client.15717 shutdown 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 msgr2=0x56261e3d86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3d86a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0068e50 tx=0x7fe6d00ad490 comp rx=0 tx=0).stop 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 shutdown_connections 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56261e43fb20 0x56261e441f40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56261e3bf2d0 0x56261e3d86a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261e3baaa0 0x56261e3d9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6c17fa700 0 client.15717 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 --2- 172.21.15.99:0/1377586400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56261e3bb3c0 0x56261e3d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 >> 172.21.15.99:0/1377586400 conn(0x56261e2e1350 msgr2=0x56261e3be930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 shutdown_connections 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.378+0000 7fe6ea7c2b80 1 -- 172.21.15.99:0/1377586400 wait complete. 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6ea7c2b80 20 client.15717 trim_cache size 0 max 16384 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 27, 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 12, 2024-05-14T17:06:57.379 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "380494ce": { 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 12, 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 146.7, 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:06:57.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:06:57.381 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe6c40b29f0 msgr2=0x7fe6c40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe6c40b29f0 0x7fe6c40b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4006fd0 tx=0x7fe6d4008040 comp rx=0 tx=0).stop 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 msgr2=0x7fe6d81a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 0x7fe6d81a5ba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000ada0 tx=0x7fe6d000c330 comp rx=0 tx=0).stop 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 shutdown_connections 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe6c40b29f0 0x7fe6c40b4eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6d8108510 0x7fe6d81a5ba0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6d8102430 0x7fe6d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 --2- 172.21.15.99:0/220908148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6d8102df0 0x7fe6d819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 >> 172.21.15.99:0/220908148 conn(0x7fe6d80793b0 msgr2=0x7fe6d8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 shutdown_connections 2024-05-14T17:06:57.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:06:57.379+0000 7fe6dff2c700 1 -- 172.21.15.99:0/220908148 wait complete. 2024-05-14T17:06:57.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:57 smithi150 ceph-mon[36204]: pgmap v102: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 966 KiB/s wr, 29 op/s 2024-05-14T17:06:57.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 37779 3.16 MB/sec warmup 69 sec latency 26697.951 ms 2024-05-14T17:06:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:57 smithi193 ceph-mon[35962]: pgmap v102: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 966 KiB/s wr, 29 op/s 2024-05-14T17:06:58.150 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '380494ce', total seconds elapsed: 32.4 2024-05-14T17:06:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 -- 172.21.15.99:0/4020194579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9904101fd0 msgr2=0x7f9904105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 --2- 172.21.15.99:0/4020194579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9904101fd0 0x7f9904105d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98f400aa10 tx=0x7f98f402b730 comp rx=0 tx=0).stop 2024-05-14T17:06:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 -- 172.21.15.99:0/4020194579 shutdown_connections 2024-05-14T17:06:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 --2- 172.21.15.99:0/4020194579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99041064b0 0x7f990410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 --2- 172.21.15.99:0/4020194579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9904101fd0 0x7f9904105d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 --2- 172.21.15.99:0/4020194579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9904101620 0x7f9904101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.308+0000 7f990b7c1640 1 -- 172.21.15.99:0/4020194579 >> 172.21.15.99:0/4020194579 conn(0x7f9904077a50 msgr2=0x7f99040fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.309+0000 7f990b7c1640 1 -- 172.21.15.99:0/4020194579 shutdown_connections 2024-05-14T17:06:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.309+0000 7f990b7c1640 1 -- 172.21.15.99:0/4020194579 wait complete. 2024-05-14T17:06:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.309+0000 7f990b7c1640 1 Processor -- start 2024-05-14T17:06:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.309+0000 7f990b7c1640 1 -- start start 2024-05-14T17:06:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.309+0000 7f990b7c1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9904101620 0x7f9904196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.309+0000 7f990b7c1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 0x7f9904197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990b7c1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f990419b7f0 0x7f990419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990b7c1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9904110b50 con 0x7f99041064b0 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990b7c1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99041109d0 con 0x7f990419b7f0 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990b7c1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9904110cd0 con 0x7f9904101620 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 0x7f9904197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 0x7f9904197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57154/0 (socket says 172.21.15.99:57154) 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 -- 172.21.15.99:0/1900963264 learned_addr learned my addr 172.21.15.99:0/1900963264 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990955d640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9904101620 0x7f9904196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9909d5e640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f990419b7f0 0x7f990419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 -- 172.21.15.99:0/1900963264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9904101620 msgr2=0x7f9904196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9904101620 0x7f9904196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 -- 172.21.15.99:0/1900963264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f990419b7f0 msgr2=0x7f990419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f990419b7f0 0x7f990419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 -- 172.21.15.99:0/1900963264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990419c2e0 con 0x7f99041064b0 2024-05-14T17:06:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9909d5e640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f990419b7f0 0x7f990419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:06:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f9908d5c640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 0x7f9904197470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f98f4030550 tx=0x7f98f4003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f98fa7fc640 1 -- 172.21.15.99:0/1900963264 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4036040 con 0x7f99041064b0 2024-05-14T17:06:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f98fa7fc640 1 -- 172.21.15.99:0/1900963264 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f4030c60 con 0x7f99041064b0 2024-05-14T17:06:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990419c570 con 0x7f99041064b0 2024-05-14T17:06:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f98fa7fc640 1 -- 172.21.15.99:0/1900963264 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4039c30 con 0x7f99041064b0 2024-05-14T17:06:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.310+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a3d30 con 0x7f99041064b0 2024-05-14T17:06:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.313+0000 7f98fa7fc640 1 -- 172.21.15.99:0/1900963264 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f98f40307b0 con 0x7f99041064b0 2024-05-14T17:06:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.313+0000 7f990e5b2740 1 Processor -- start 2024-05-14T17:06:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.313+0000 7f990e5b2740 1 -- start start 2024-05-14T17:06:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990e5b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5602b3620c30 con 0x5602b36ae210 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990e5b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5602b36107d0 con 0x5602b36ad8f0 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990e5b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602b3614810 con 0x5602b36b8ad0 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990955d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990955d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57168/0 (socket says 172.21.15.99:57168) 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990955d640 1 -- 172.21.15.99:0/4073483287 learned_addr learned my addr 172.21.15.99:0/4073483287 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9908d5c640 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 -- 172.21.15.99:0/4073483287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 msgr2=0x5602b36baec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36baec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 -- 172.21.15.99:0/4073483287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 msgr2=0x5602b36b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 -- 172.21.15.99:0/4073483287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b3325210 con 0x5602b36ad8f0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990955d640 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f98fa7fc640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f98f00b2a60 0x7f98f00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f9909d5e640 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36adcd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f98fc010690 tx=0x7f98fc010b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924d28cf46325c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f990955d640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f98f00b2a60 0x7f98f00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f98fa7fc640 1 -- 172.21.15.99:0/1900963264 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f98f4104930 con 0x7f99041064b0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.314+0000 7f98ef7fe640 1 -- 172.21.15.99:0/4073483287 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc004b20 con 0x5602b36ad8f0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f98ef7fe640 1 -- 172.21.15.99:0/4073483287 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc00ed80 con 0x5602b36ad8f0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f98ef7fe640 1 -- 172.21.15.99:0/4073483287 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc005740 con 0x5602b36ad8f0 2024-05-14T17:06:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 -- 172.21.15.99:0/4073483287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 msgr2=0x5602b36adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36adcd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f98fc010690 tx=0x7f98fc010b50 comp rx=0 tx=0).stop 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990955d640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f98f00b2a60 0x7f98f00b4f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5602b36aee90 tx=0x7f990000a2d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 -- 172.21.15.99:0/4073483287 shutdown_connections 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4073483287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36adcd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 -- 172.21.15.99:0/4073483287 >> 172.21.15.99:0/4073483287 conn(0x5602b36a9e00 msgr2=0x5602b36aa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 -- 172.21.15.99:0/4073483287 shutdown_connections 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.315+0000 7f990e5b2740 1 -- 172.21.15.99:0/4073483287 wait complete. 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 -- start start 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 Processor -- start 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 0x5602b36b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5602b32f6790 con 0x5602b36ad8f0 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5602b35cc890 con 0x5602b36ae210 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602b36bb7a0 con 0x5602b36b8ad0 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990e5b2740 10 client.0 _ll_register_callbacks cb 0x5602b35cfd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990955d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 0x5602b36b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f9909d5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990955d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 0x5602b36b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57182/0 (socket says 172.21.15.99:57182) 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f990955d640 1 -- 172.21.15.99:0/420129789 learned_addr learned my addr 172.21.15.99:0/420129789 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.316+0000 7f9908d5c640 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990955d640 1 -- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 msgr2=0x5602b36b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990955d640 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990955d640 1 -- 172.21.15.99:0/420129789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 msgr2=0x5602b36b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990955d640 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990955d640 1 -- 172.21.15.99:0/420129789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b33f1620 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990955d640 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 0x5602b36b8470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f990000fcd0 tx=0x7f9900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6040bccbf354604c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f98ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f98ecff9640 1 -- 172.21.15.99:0/420129789 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990000fdc0 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f98ecff9640 1 -- 172.21.15.99:0/420129789 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f9900016030 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98fc00ed80 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f98ecff9640 1 -- 172.21.15.99:0/420129789 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f990001c8a0 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990e5b2740 10 client.15726 fetch_fsmap learned FSMap version 27 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602b36c72b0 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.317+0000 7f98ecff9640 1 -- 172.21.15.99:0/420129789 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc00ed80 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f98ecff9640 1 -- 172.21.15.99:0/420129789 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9900027960 con 0x5602b36ad8f0 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f990e5b2740 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f990e5b2740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f990e5b2740 20 client.15726 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f990e5b2740 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b36c87e0 0x5602b36cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f990e5b2740 4 client.15726 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602b3622b80 con 0x5602b36c87e0 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f9908d5c640 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b36c87e0 0x5602b36cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f98ecff9640 10 client.15726 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.318+0000 7f9908d5c640 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b36c87e0 0x5602b36cabd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66df64520cec731 server_cookie=aee246ddd7aed6c0 in_seq=0 out_seq=0 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f98ecff9640 1 -- 172.21.15.99:0/420129789 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5602b3622b80 con 0x5602b36c87e0 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f98ecff9640 10 client.15726 handle_command_reply: tid=0 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 client.15726 shutdown 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 msgr2=0x5602b36b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 0x5602b36b8470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f990000fcd0 tx=0x7f9900004730 comp rx=0 tx=0).stop 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 shutdown_connections 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b36c87e0 0x5602b36cabd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 --2- 172.21.15.99:0/420129789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ad8f0 0x5602b36b8470 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f98ecff9640 0 client.15726 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 >> 172.21.15.99:0/420129789 conn(0x5602b36a9e00 msgr2=0x5602b36a9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 shutdown_connections 2024-05-14T17:06:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.319+0000 7f990e5b2740 1 -- 172.21.15.99:0/420129789 wait complete. 2024-05-14T17:06:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.320+0000 7f990e5b2740 20 client.15726 trim_cache size 0 max 16384 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.335+0000 7f990e5b2740 1 Processor -- start 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 -- start start 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aeb40 con 0x5602b36ae210 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ac410 con 0x5602b36ad8f0 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990e5b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aa280 con 0x5602b36b8ad0 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57184/0 (socket says 172.21.15.99:57184) 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 -- 172.21.15.99:0/1628237433 learned_addr learned my addr 172.21.15.99:0/1628237433 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9909d5e640 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990955d640 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 -- 172.21.15.99:0/1628237433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 msgr2=0x5602b36b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 -- 172.21.15.99:0/1628237433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 msgr2=0x5602b36d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 -- 172.21.15.99:0/1628237433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b36b6070 con 0x5602b36ae210 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f990955d640 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f9908d5c640 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36d0a70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f98f4005ab0 tx=0x7f98f4106040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84bad215f7aea01 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f98eeffd640 1 -- 172.21.15.99:0/1628237433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40cb590 con 0x5602b36ae210 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.336+0000 7f98eeffd640 1 -- 172.21.15.99:0/1628237433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f4030aa0 con 0x5602b36ae210 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f98eeffd640 1 -- 172.21.15.99:0/1628237433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40c73d0 con 0x5602b36ae210 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 -- 172.21.15.99:0/1628237433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 msgr2=0x5602b36d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36d0a70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f98f4005ab0 tx=0x7f98f4106040 comp rx=0 tx=0).stop 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 -- 172.21.15.99:0/1628237433 shutdown_connections 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36b8ad0 0x5602b36b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36ae210 0x5602b36d0a70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 --2- 172.21.15.99:0/1628237433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ad8f0 0x5602b36d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 -- 172.21.15.99:0/1628237433 >> 172.21.15.99:0/1628237433 conn(0x5602b36a9e00 msgr2=0x5602b36c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 -- 172.21.15.99:0/1628237433 shutdown_connections 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.337+0000 7f990e5b2740 1 -- 172.21.15.99:0/1628237433 wait complete. 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 -- start start 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 Processor -- start 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36ad8f0 0x5602b36be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 0x5602b36bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aa8d0 con 0x5602b36b8ad0 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99041abd40 con 0x5602b36ae210 2024-05-14T17:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99041aafa0 con 0x5602b36ad8f0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990e5b2740 10 client.0 _ll_register_callbacks cb 0x7f990000f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 0x5602b36bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 0x5602b36bf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57200/0 (socket says 172.21.15.99:57200) 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 -- 172.21.15.99:0/4091306103 learned_addr learned my addr 172.21.15.99:0/4091306103 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f990955d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36ad8f0 0x5602b36be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9908d5c640 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 -- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36ad8f0 msgr2=0x5602b36be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36ad8f0 0x5602b36be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 -- 172.21.15.99:0/4091306103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 msgr2=0x5602b36bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.338+0000 7f9909d5e640 1 -- 172.21.15.99:0/4091306103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b3732220 con 0x5602b36b8ad0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f9909d5e640 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 0x5602b36bf170 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0098b0 tx=0x7f98fc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2161cad0310db5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f98ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f98ecff9640 1 -- 172.21.15.99:0/4091306103 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00ece0 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f98ecff9640 1 -- 172.21.15.99:0/4091306103 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f98fc0049a0 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602b361c360 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f98ecff9640 1 -- 172.21.15.99:0/4091306103 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98fc036050 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 10 client.15732 fetch_fsmap learned FSMap version 27 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602b3733510 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f98ecff9640 1 -- 172.21.15.99:0/4091306103 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5602b361c360 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f98ecff9640 1 -- 172.21.15.99:0/4091306103 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f98fc00f040 con 0x5602b36b8ad0 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 20 client.15732 populate_metadata read hostname 'smithi099' 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b3733940 0x5602b3735d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 4 client.15732 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.339+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602b35965f0 con 0x5602b3733940 2024-05-14T17:06:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.340+0000 7f990955d640 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b3733940 0x5602b3735d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.340+0000 7f990955d640 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b3733940 0x5602b3735d30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e07c4c8cff1e0029 server_cookie=b52096610861e205 in_seq=0 out_seq=0 2024-05-14T17:06:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.340+0000 7f98ecff9640 10 client.15732 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f98ecff9640 1 -- 172.21.15.99:0/4091306103 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5602b35965f0 con 0x5602b3733940 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f98ecff9640 10 client.15732 handle_command_reply: tid=0 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 client.15732 shutdown 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 msgr2=0x5602b36bf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 0x5602b36bf170 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0098b0 tx=0x7f98fc010da0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 shutdown_connections 2024-05-14T17:06:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5602b3733940 0x5602b3735d30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5602b36b8ad0 0x5602b36bf170 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5602b36ae210 0x5602b36bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 --2- 172.21.15.99:0/4091306103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602b36ad8f0 0x5602b36be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 >> 172.21.15.99:0/4091306103 conn(0x5602b36a9e00 msgr2=0x5602b36c3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f98ecff9640 0 client.15732 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 shutdown_connections 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 1 -- 172.21.15.99:0/4091306103 wait complete. 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.471+0000 7f990e5b2740 20 client.15732 trim_cache size 0 max 16384 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:06:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:06:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f98f00b2a60 msgr2=0x7f98f00b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f98f00b2a60 0x7f98f00b4f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5602b36aee90 tx=0x7f990000a2d0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 msgr2=0x7f9904197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 0x7f9904197470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f98f4030550 tx=0x7f98f4003c00 comp rx=0 tx=0).stop 2024-05-14T17:06:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 shutdown_connections 2024-05-14T17:06:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f98f00b2a60 0x7f98f00b4f20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f990419b7f0 0x7f990419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99041064b0 0x7f9904197470 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 --2- 172.21.15.99:0/1900963264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9904101620 0x7f9904196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 >> 172.21.15.99:0/1900963264 conn(0x7f9904077a50 msgr2=0x7f9904104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 shutdown_connections 2024-05-14T17:06:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:06:58.472+0000 7f990b7c1640 1 -- 172.21.15.99:0/1900963264 wait complete. 2024-05-14T17:06:58.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 39845 3.31 MB/sec warmup 70 sec latency 38.609 ms 2024-05-14T17:06:59.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:06:59 smithi150 ceph-mon[36204]: pgmap v103: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 966 KiB/s wr, 29 op/s 2024-05-14T17:06:59.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:06:59 smithi099 ceph-mon[31851]: pgmap v103: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 966 KiB/s wr, 29 op/s 2024-05-14T17:06:59.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 42962 3.53 MB/sec warmup 71 sec latency 6.932 ms 2024-05-14T17:06:59.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:06:59 smithi193 ceph-mon[35962]: pgmap v103: 97 pgs: 97 active+clean; 38 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 966 KiB/s wr, 29 op/s 2024-05-14T17:07:00.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 45830 3.66 MB/sec warmup 72 sec latency 16.988 ms 2024-05-14T17:07:01.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:01 smithi150 ceph-mon[36204]: pgmap v104: 97 pgs: 97 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-14T17:07:01.513 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:01 smithi099 ceph-mon[31851]: pgmap v104: 97 pgs: 97 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-14T17:07:01.520 INFO:tasks.workunit.client.0.smithi099.stdout: 1 47055 3.70 MB/sec warmup 73 sec latency 24.135 ms 2024-05-14T17:07:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:01 smithi193 ceph-mon[35962]: pgmap v104: 97 pgs: 97 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-14T17:07:02.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 50083 3.90 MB/sec warmup 74 sec latency 13.362 ms 2024-05-14T17:07:03.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:03 smithi150 ceph-mon[36204]: pgmap v105: 97 pgs: 97 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 80 op/s 2024-05-14T17:07:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:03 smithi099 ceph-mon[31851]: pgmap v105: 97 pgs: 97 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 80 op/s 2024-05-14T17:07:03.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 53235 4.08 MB/sec warmup 75 sec latency 17.755 ms 2024-05-14T17:07:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:03 smithi193 ceph-mon[35962]: pgmap v105: 97 pgs: 97 active+clean; 27 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 80 op/s 2024-05-14T17:07:04.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 55705 4.15 MB/sec warmup 76 sec latency 12.972 ms 2024-05-14T17:07:05.420 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:05 smithi099 ceph-mon[31851]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-05-14T17:07:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:05 smithi150 ceph-mon[36204]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-05-14T17:07:05.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 57968 4.27 MB/sec warmup 77 sec latency 12.952 ms 2024-05-14T17:07:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:05 smithi193 ceph-mon[35962]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-05-14T17:07:06.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 60874 4.44 MB/sec warmup 78 sec latency 20.050 ms 2024-05-14T17:07:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:07 smithi150 ceph-mon[36204]: pgmap v107: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-05-14T17:07:07.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:07 smithi099 ceph-mon[31851]: pgmap v107: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-05-14T17:07:07.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 64010 4.61 MB/sec warmup 79 sec latency 12.883 ms 2024-05-14T17:07:07.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:07 smithi193 ceph-mon[35962]: pgmap v107: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-05-14T17:07:08.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 66770 4.71 MB/sec warmup 80 sec latency 12.439 ms 2024-05-14T17:07:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:09 smithi150 ceph-mon[36204]: pgmap v108: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-05-14T17:07:09.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:09 smithi099 ceph-mon[31851]: pgmap v108: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-05-14T17:07:09.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 69106 4.81 MB/sec warmup 81 sec latency 13.662 ms 2024-05-14T17:07:09.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:09 smithi193 ceph-mon[35962]: pgmap v108: 97 pgs: 97 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-05-14T17:07:10.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 70968 4.90 MB/sec warmup 82 sec latency 521.642 ms 2024-05-14T17:07:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:11 smithi150 ceph-mon[36204]: pgmap v109: 97 pgs: 97 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-05-14T17:07:11.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:11 smithi099 ceph-mon[31851]: pgmap v109: 97 pgs: 97 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-05-14T17:07:11.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 70968 4.84 MB/sec warmup 83 sec latency 1521.687 ms 2024-05-14T17:07:11.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:11 smithi193 ceph-mon[35962]: pgmap v109: 97 pgs: 97 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-05-14T17:07:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 -- 172.21.15.99:0/1973535111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28101970 msgr2=0x7f2c28101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/1973535111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28101970 0x7f2c28101df0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009930 tx=0x7f2c1002a420 comp rx=0 tx=0).stop 2024-05-14T17:07:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 -- 172.21.15.99:0/1973535111 shutdown_connections 2024-05-14T17:07:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/1973535111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c28102330 0x7f2c2810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/1973535111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28101970 0x7f2c28101df0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/1973535111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c28107980 0x7f2c28107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 -- 172.21.15.99:0/1973535111 >> 172.21.15.99:0/1973535111 conn(0x7f2c28077cd0 msgr2=0x7f2c280780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 -- 172.21.15.99:0/1973535111 shutdown_connections 2024-05-14T17:07:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.571+0000 7f2c2cb78640 1 -- 172.21.15.99:0/1973535111 wait complete. 2024-05-14T17:07:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 Processor -- start 2024-05-14T17:07:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 -- start start 2024-05-14T17:07:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c28101970 0x7f2c28199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 0x7f2c281996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c28107980 0x7f2c2819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28111c10 con 0x7f2c28102330 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28111a90 con 0x7f2c28101970 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.572+0000 7f2c2cb78640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28111d90 con 0x7f2c28107980 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 0x7f2c281996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 0x7f2c281996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57100/0 (socket says 172.21.15.99:57100) 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 -- 172.21.15.99:0/4078404358 learned_addr learned my addr 172.21.15.99:0/4078404358 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c2659c640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c28101970 0x7f2c28199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c28107980 0x7f2c2819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 -- 172.21.15.99:0/4078404358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c28107980 msgr2=0x7f2c2819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c28107980 0x7f2c2819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 -- 172.21.15.99:0/4078404358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c28101970 msgr2=0x7f2c28199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c28101970 0x7f2c28199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 -- 172.21.15.99:0/4078404358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2819e110 con 0x7f2c28102330 2024-05-14T17:07:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 0x7f2c281996b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2c10004770 tx=0x7f2c100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1f7fe640 1 -- 172.21.15.99:0/4078404358 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1003f070 con 0x7f2c28102330 2024-05-14T17:07:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1f7fe640 1 -- 172.21.15.99:0/4078404358 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1003b460 con 0x7f2c28102330 2024-05-14T17:07:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c1f7fe640 1 -- 172.21.15.99:0/4078404358 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10035070 con 0x7f2c28102330 2024-05-14T17:07:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2819e3a0 con 0x7f2c28102330 2024-05-14T17:07:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.573+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281a5c90 con 0x7f2c28102330 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.576+0000 7f2c2f96f740 1 Processor -- start 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 -- start start 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c1f7fe640 1 -- 172.21.15.99:0/4078404358 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c1003b8d0 con 0x7f2c28102330 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab716cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab721590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab721ad0 0x5651ab723ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5651ab689c30 con 0x5651ab7168f0 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651ab6797d0 con 0x5651ab721ad0 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651ab67d810 con 0x5651ab717210 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab716cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab716cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57110/0 (socket says 172.21.15.99:57110) 2024-05-14T17:07:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 -- 172.21.15.99:0/3116381021 learned_addr learned my addr 172.21.15.99:0/3116381021 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c2659c640 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab721590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab721ad0 0x5651ab723ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 -- 172.21.15.99:0/3116381021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 msgr2=0x5651ab721590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab721590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 -- 172.21.15.99:0/3116381021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab721ad0 msgr2=0x5651ab723ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab721ad0 0x5651ab723ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.577+0000 7f2c26d9d640 1 -- 172.21.15.99:0/3116381021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ab38e210 con 0x5651ab7168f0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab716cd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009ee0 tx=0x7f2c20010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72294ce42525bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c1f7fe640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2c140b2a10 0x7f2c140b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c1cff9640 1 -- 172.21.15.99:0/3116381021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20016070 con 0x5651ab7168f0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c2659c640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2c140b2a10 0x7f2c140b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c1cff9640 1 -- 172.21.15.99:0/3116381021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c20004d20 con 0x5651ab7168f0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c1cff9640 1 -- 172.21.15.99:0/3116381021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20012740 con 0x5651ab7168f0 2024-05-14T17:07:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.578+0000 7f2c1f7fe640 1 -- 172.21.15.99:0/4078404358 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2c101029a0 con 0x7f2c28102330 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- 172.21.15.99:0/3116381021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 msgr2=0x5651ab716cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2659c640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2c140b2a10 0x7f2c140b4ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5651ab717e90 tx=0x7f2c1800a2d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab716cd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009ee0 tx=0x7f2c20010740 comp rx=0 tx=0).stop 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- 172.21.15.99:0/3116381021 shutdown_connections 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab721ad0 0x5651ab723ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab721590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/3116381021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab716cd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- 172.21.15.99:0/3116381021 >> 172.21.15.99:0/3116381021 conn(0x5651ab712e00 msgr2=0x5651ab7131e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- 172.21.15.99:0/3116381021 shutdown_connections 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- 172.21.15.99:0/3116381021 wait complete. 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- start start 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 Processor -- start 2024-05-14T17:07:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab7213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab71e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab71ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5651ab35f790 con 0x5651ab7168f0 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651ab635890 con 0x5651ab717210 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.579+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651ab7247a0 con 0x5651ab721ad0 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2f96f740 10 client.0 _ll_register_callbacks cb 0x5651ab638d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab7213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab7213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57118/0 (socket says 172.21.15.99:57118) 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 -- 172.21.15.99:0/404533622 learned_addr learned my addr 172.21.15.99:0/404533622 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab71ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab71e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 -- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 msgr2=0x5651ab71ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab71ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 -- 172.21.15.99:0/404533622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 msgr2=0x5651ab71e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab71e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 -- 172.21.15.99:0/404533622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ab45a620 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2659c640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab7213e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c1801cc10 tx=0x7f2c18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468d3f5aac40eea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c0dffb640 1 -- 172.21.15.99:0/404533622 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1802c060 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c0dffb640 1 -- 172.21.15.99:0/404533622 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2c18016030 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c20004d20 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c0dffb640 1 -- 172.21.15.99:0/404533622 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c1801cc40 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2f96f740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.580+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651ab7301d0 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c0dffb640 1 -- 172.21.15.99:0/404533622 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c20004d20 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c0dffb640 1 -- 172.21.15.99:0/404533622 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2c1801a610 con 0x5651ab7168f0 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c2f96f740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c2f96f740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c2f96f740 20 client.15744 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7316e0 0x5651ab733ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c2f96f740 4 client.15744 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ab68bb80 con 0x5651ab7316e0 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.581+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7316e0 0x5651ab733ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7316e0 0x5651ab733ad0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f4d7bf9b2e01fc server_cookie=18bab196aed7f5ab in_seq=0 out_seq=0 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c0dffb640 10 client.15744 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c0dffb640 1 -- 172.21.15.99:0/404533622 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651ab68bb80 con 0x5651ab7316e0 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c0dffb640 10 client.15744 handle_command_reply: tid=0 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c2f96f740 1 client.15744 shutdown 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 msgr2=0x5651ab7213e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab7213e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c1801cc10 tx=0x7f2c18004730 comp rx=0 tx=0).stop 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.582+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 shutdown_connections 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7316e0 0x5651ab733ad0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab71ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab71e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/404533622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab7213e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c0dffb640 0 client.15744 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 >> 172.21.15.99:0/404533622 conn(0x5651ab712e00 msgr2=0x5651ab712100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 shutdown_connections 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 1 -- 172.21.15.99:0/404533622 wait complete. 2024-05-14T17:07:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.583+0000 7f2c2f96f740 20 client.15744 trim_cache size 0 max 16384 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 Processor -- start 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 -- start start 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab72d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab7393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab739900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b0df0 con 0x5651ab7168f0 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ae6c0 con 0x5651ab717210 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ac510 con 0x5651ab721ad0 2024-05-14T17:07:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab72d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab72d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57130/0 (socket says 172.21.15.99:57130) 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 -- 172.21.15.99:0/2125119505 learned_addr learned my addr 172.21.15.99:0/2125119505 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab7393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab739900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 -- 172.21.15.99:0/2125119505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 msgr2=0x5651ab739900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab739900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 -- 172.21.15.99:0/2125119505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 msgr2=0x5651ab7393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.598+0000 7f2c2659c640 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab7393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2659c640 1 -- 172.21.15.99:0/2125119505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ab71d680 con 0x5651ab7168f0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2659c640 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab72d670 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2c18017040 tx=0x7f2c1800a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2802c30d7a9cf6be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c0ffff640 1 -- 172.21.15.99:0/2125119505 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1803f070 con 0x5651ab7168f0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c0ffff640 1 -- 172.21.15.99:0/2125119505 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1801cde0 con 0x5651ab7168f0 2024-05-14T17:07:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c0ffff640 1 -- 172.21.15.99:0/2125119505 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800fcb0 con 0x5651ab7168f0 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 -- 172.21.15.99:0/2125119505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 msgr2=0x5651ab72d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab72d670 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2c18017040 tx=0x7f2c1800a760 comp rx=0 tx=0).stop 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 -- 172.21.15.99:0/2125119505 shutdown_connections 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab721ad0 0x5651ab739900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab717210 0x5651ab7393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/2125119505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab7168f0 0x5651ab72d670 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 -- 172.21.15.99:0/2125119505 >> 172.21.15.99:0/2125119505 conn(0x5651ab712e00 msgr2=0x5651ab712100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 -- 172.21.15.99:0/2125119505 shutdown_connections 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.599+0000 7f2c2f96f740 1 -- 172.21.15.99:0/2125119505 wait complete. 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 -- start start 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 Processor -- start 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab7168f0 0x5651ab727610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab727b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 0x5651ab728090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281acb80 con 0x5651ab721ad0 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281adff0 con 0x5651ab7168f0 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ad250 con 0x5651ab717210 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2f96f740 10 client.0 _ll_register_callbacks cb 0x7f2c1801c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 0x5651ab728090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 0x5651ab728090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57144/0 (socket says 172.21.15.99:57144) 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 -- 172.21.15.99:0/1599537292 learned_addr learned my addr 172.21.15.99:0/1599537292 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c1ffff640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab727b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 -- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 msgr2=0x5651ab727b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab727b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 -- 172.21.15.99:0/1599537292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab7168f0 msgr2=0x5651ab727610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c2659c640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab7168f0 0x5651ab727610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab7168f0 0x5651ab727610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.600+0000 7f2c26d9d640 1 -- 172.21.15.99:0/1599537292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ab79f3e0 con 0x5651ab721ad0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c26d9d640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 0x5651ab728090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009ee0 tx=0x7f2c20004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b982fab3a907e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c0dffb640 1 -- 172.21.15.99:0/1599537292 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20029060 con 0x5651ab721ad0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c0dffb640 1 -- 172.21.15.99:0/1599537292 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2c20024680 con 0x5651ab721ad0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c1801cde0 con 0x5651ab721ad0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c0dffb640 1 -- 172.21.15.99:0/1599537292 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c20016040 con 0x5651ab721ad0 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 10 client.15750 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5651ab7a06f0 con 0x5651ab721ad0 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c0dffb640 1 -- 172.21.15.99:0/1599537292 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1801cde0 con 0x5651ab721ad0 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c0dffb640 1 -- 172.21.15.99:0/1599537292 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2c2000f040 con 0x5651ab721ad0 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 20 client.15750 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7a0bd0 0x5651ab7a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 4 client.15750 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.601+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651ab685360 con 0x5651ab7a0bd0 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.602+0000 7f2c2659c640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7a0bd0 0x5651ab7a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.602+0000 7f2c0dffb640 10 client.15750 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:11.602+0000 7f2c2659c640 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7a0bd0 0x5651ab7a2ff0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe118fe06dc6b3 server_cookie=3b8e54ea8c994bac in_seq=0 out_seq=0 2024-05-14T17:07:12.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.80 MB/sec warmup 84 sec latency 1699.606 ms 2024-05-14T17:07:13.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:13 smithi150 ceph-mon[36204]: pgmap v110: 97 pgs: 97 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-05-14T17:07:13.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:13 smithi099 ceph-mon[31851]: pgmap v110: 97 pgs: 97 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-05-14T17:07:13.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.75 MB/sec warmup 85 sec latency 1375.699 ms 2024-05-14T17:07:13.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:13 smithi193 ceph-mon[35962]: pgmap v110: 97 pgs: 97 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-05-14T17:07:14.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.69 MB/sec warmup 86 sec latency 2375.794 ms 2024-05-14T17:07:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:15 smithi150 ceph-mon[36204]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 320 op/s 2024-05-14T17:07:15.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:15 smithi099 ceph-mon[31851]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 320 op/s 2024-05-14T17:07:15.521 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.64 MB/sec warmup 87 sec latency 3375.909 ms 2024-05-14T17:07:15.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:15 smithi193 ceph-mon[35962]: pgmap v111: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 320 op/s 2024-05-14T17:07:16.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.58 MB/sec warmup 88 sec latency 4376.021 ms 2024-05-14T17:07:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:17 smithi150 ceph-mon[36204]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-14T17:07:17.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:17 smithi099 ceph-mon[31851]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-14T17:07:17.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.53 MB/sec warmup 89 sec latency 5376.079 ms 2024-05-14T17:07:17.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:17 smithi193 ceph-mon[35962]: pgmap v112: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-14T17:07:18.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.48 MB/sec warmup 90 sec latency 6376.198 ms 2024-05-14T17:07:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:19 smithi150 ceph-mon[36204]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-14T17:07:19.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:19 smithi099 ceph-mon[31851]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-14T17:07:19.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 71556 4.43 MB/sec warmup 91 sec latency 7376.292 ms 2024-05-14T17:07:19.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:19 smithi193 ceph-mon[35962]: pgmap v113: 97 pgs: 97 active+clean; 45 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c0dffb640 1 -- 172.21.15.99:0/1599537292 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5651ab685360 con 0x5651ab7a0bd0 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c0dffb640 10 client.15750 handle_command_reply: tid=0 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 client.15750 shutdown 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 msgr2=0x5651ab728090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 0x5651ab728090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009ee0 tx=0x7f2c20004c10 comp rx=0 tx=0).stop 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 shutdown_connections 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5651ab7a0bd0 0x5651ab7a2ff0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651ab721ad0 0x5651ab728090 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651ab717210 0x5651ab727b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 --2- 172.21.15.99:0/1599537292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651ab7168f0 0x5651ab727610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c0dffb640 0 client.15750 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 >> 172.21.15.99:0/1599537292 conn(0x5651ab712e00 msgr2=0x5651ab72c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 shutdown_connections 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.615+0000 7f2c2f96f740 1 -- 172.21.15.99:0/1599537292 wait complete. 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.616+0000 7f2c2f96f740 20 client.15750 trim_cache size 0 max 16384 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:07:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:07:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.616+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2c140b2a10 msgr2=0x7f2c140b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.616+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2c140b2a10 0x7f2c140b4ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5651ab717e90 tx=0x7f2c1800a2d0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.616+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 msgr2=0x7f2c281996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.616+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 0x7f2c281996b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2c10004770 tx=0x7f2c100042f0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 shutdown_connections 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2c140b2a10 0x7f2c140b4ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c28107980 0x7f2c2819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c28102330 0x7f2c281996b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 --2- 172.21.15.99:0/4078404358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c28101970 0x7f2c28199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 >> 172.21.15.99:0/4078404358 conn(0x7f2c28077cd0 msgr2=0x7f2c2806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 shutdown_connections 2024-05-14T17:07:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:19.617+0000 7f2c2cb78640 1 -- 172.21.15.99:0/4078404358 wait complete. 2024-05-14T17:07:20.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 74365 4.54 MB/sec warmup 92 sec latency 7461.928 ms 2024-05-14T17:07:21.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:21 smithi150 ceph-mon[36204]: pgmap v114: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 202 op/s 2024-05-14T17:07:21.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:21 smithi099 ceph-mon[31851]: pgmap v114: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 202 op/s 2024-05-14T17:07:21.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 76238 4.60 MB/sec warmup 93 sec latency 18.369 ms 2024-05-14T17:07:21.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:21 smithi193 ceph-mon[35962]: pgmap v114: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 202 op/s 2024-05-14T17:07:22.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 78820 4.70 MB/sec warmup 94 sec latency 11.816 ms 2024-05-14T17:07:23.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:23 smithi150 ceph-mon[36204]: pgmap v115: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 385 KiB/s wr, 111 op/s 2024-05-14T17:07:23.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:23 smithi099 ceph-mon[31851]: pgmap v115: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 385 KiB/s wr, 111 op/s 2024-05-14T17:07:23.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 81789 4.84 MB/sec warmup 95 sec latency 13.597 ms 2024-05-14T17:07:23.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:23 smithi193 ceph-mon[35962]: pgmap v115: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 385 KiB/s wr, 111 op/s 2024-05-14T17:07:24.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 84053 4.87 MB/sec warmup 96 sec latency 13.649 ms 2024-05-14T17:07:25.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:25 smithi150 ceph-mon[36204]: pgmap v116: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-05-14T17:07:25.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:25 smithi099 ceph-mon[31851]: pgmap v116: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-05-14T17:07:25.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 86157 4.96 MB/sec warmup 97 sec latency 15.931 ms 2024-05-14T17:07:25.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:25 smithi193 ceph-mon[35962]: pgmap v116: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-05-14T17:07:26.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 88985 5.09 MB/sec warmup 98 sec latency 13.601 ms 2024-05-14T17:07:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:27 smithi150 ceph-mon[36204]: pgmap v117: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 710 KiB/s wr, 114 op/s 2024-05-14T17:07:27.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:27 smithi099 ceph-mon[31851]: pgmap v117: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 710 KiB/s wr, 114 op/s 2024-05-14T17:07:27.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 91162 5.11 MB/sec warmup 99 sec latency 13.354 ms 2024-05-14T17:07:27.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:27 smithi193 ceph-mon[35962]: pgmap v117: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 710 KiB/s wr, 114 op/s 2024-05-14T17:07:28.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 93143 5.19 MB/sec warmup 100 sec latency 29.348 ms 2024-05-14T17:07:29.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:29 smithi150 ceph-mon[36204]: pgmap v118: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 710 KiB/s wr, 114 op/s 2024-05-14T17:07:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:29 smithi099 ceph-mon[31851]: pgmap v118: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 710 KiB/s wr, 114 op/s 2024-05-14T17:07:29.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 96053 5.32 MB/sec warmup 101 sec latency 17.360 ms 2024-05-14T17:07:29.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:29 smithi193 ceph-mon[35962]: pgmap v118: 97 pgs: 97 active+clean; 36 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 710 KiB/s wr, 114 op/s 2024-05-14T17:07:30.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 98271 5.34 MB/sec warmup 102 sec latency 16.037 ms 2024-05-14T17:07:31.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:31 smithi150 ceph-mon[36204]: pgmap v119: 97 pgs: 97 active+clean; 47 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-05-14T17:07:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:31 smithi099 ceph-mon[31851]: pgmap v119: 97 pgs: 97 active+clean; 47 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-05-14T17:07:31.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 100469 5.42 MB/sec warmup 103 sec latency 12.738 ms 2024-05-14T17:07:31.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:31 smithi193 ceph-mon[35962]: pgmap v119: 97 pgs: 97 active+clean; 47 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-05-14T17:07:32.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 103184 5.53 MB/sec warmup 104 sec latency 21.309 ms 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 -- 172.21.15.99:0/3823708145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c1033f0 msgr2=0x7f0b8c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3823708145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c1033f0 0x7f0b8c10f900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009cd0 tx=0x7f0b8402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 -- 172.21.15.99:0/3823708145 shutdown_connections 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3823708145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c1033f0 0x7f0b8c10f900 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3823708145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 0x7f0b8c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3823708145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c108a40 0x7f0b8c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 -- 172.21.15.99:0/3823708145 >> 172.21.15.99:0/3823708145 conn(0x7f0b8c0fe500 msgr2=0x7f0b8c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 -- 172.21.15.99:0/3823708145 shutdown_connections 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 -- 172.21.15.99:0/3823708145 wait complete. 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.715+0000 7f0b9128c640 1 Processor -- start 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 -- start start 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 0x7f0b8c074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c1033f0 0x7f0b8c075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 0x7f0b8c0757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112d00 con 0x7f0b8c108a40 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112b80 con 0x7f0b8c1033f0 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b9128c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112e80 con 0x7f0b8c102a30 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 0x7f0b8c0757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 0x7f0b8c074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 0x7f0b8c0757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39414/0 (socket says 172.21.15.99:39414) 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 -- 172.21.15.99:0/3946334614 learned_addr learned my addr 172.21.15.99:0/3946334614 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c1033f0 0x7f0b8c075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 -- 172.21.15.99:0/3946334614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 msgr2=0x7f0b8c074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 0x7f0b8c074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 -- 172.21.15.99:0/3946334614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c1033f0 msgr2=0x7f0b8c075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c1033f0 0x7f0b8c075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 -- 172.21.15.99:0/3946334614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1a76a0 con 0x7f0b8c108a40 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8affd640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 0x7f0b8c074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.716+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 0x7f0b8c0757d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0b84033040 tx=0x7f0b84003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.717+0000 7f0b73fff640 1 -- 172.21.15.99:0/3946334614 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84004040 con 0x7f0b8c108a40 2024-05-14T17:07:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.717+0000 7f0b73fff640 1 -- 172.21.15.99:0/3946334614 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b84004690 con 0x7f0b8c108a40 2024-05-14T17:07:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.717+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1a7930 con 0x7f0b8c108a40 2024-05-14T17:07:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.717+0000 7f0b73fff640 1 -- 172.21.15.99:0/3946334614 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84030e20 con 0x7f0b8c108a40 2024-05-14T17:07:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.717+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a7e80 con 0x7f0b8c108a40 2024-05-14T17:07:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b73fff640 1 -- 172.21.15.99:0/3946334614 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0b840305a0 con 0x7f0b8c108a40 2024-05-14T17:07:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 Processor -- start 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 -- start start 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761aa5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761ab2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e761a18c30 con 0x55e761aa58f0 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e761a087d0 con 0x55e761aa6210 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e761a0c810 con 0x55e761ab0ad0 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b8a7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761aa5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b8a7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761aa5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39418/0 (socket says 172.21.15.99:39418) 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.719+0000 7f0b8a7fc640 1 -- 172.21.15.99:0/3364882010 learned_addr learned my addr 172.21.15.99:0/3364882010 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8affd640 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761ab2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 -- 172.21.15.99:0/3364882010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 msgr2=0x55e761ab2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761ab2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 -- 172.21.15.99:0/3364882010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 msgr2=0x55e761ab0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 -- 172.21.15.99:0/3364882010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76171d210 con 0x55e761aa58f0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b73fff640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0b740b2a10 0x7f0b740b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761aa5cd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009710 tx=0x7f0b80009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3dfab66d389812 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b8affd640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0b740b2a10 0x7f0b740b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b73fff640 1 -- 172.21.15.99:0/3946334614 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f0b8402e030 con 0x7f0b8c108a40 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b717fa640 1 -- 172.21.15.99:0/3364882010 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80013d70 con 0x55e761aa58f0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b717fa640 1 -- 172.21.15.99:0/3364882010 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80013120 con 0x55e761aa58f0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.720+0000 7f0b717fa640 1 -- 172.21.15.99:0/3364882010 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000fe40 con 0x55e761aa58f0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 -- 172.21.15.99:0/3364882010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 msgr2=0x55e761aa5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b8affd640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0b740b2a10 0x7f0b740b4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55e761aa6e90 tx=0x7f0b7c00b390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761aa5cd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009710 tx=0x7f0b80009780 comp rx=0 tx=0).stop 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 -- 172.21.15.99:0/3364882010 shutdown_connections 2024-05-14T17:07:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761ab2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/3364882010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761aa5cd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 -- 172.21.15.99:0/3364882010 >> 172.21.15.99:0/3364882010 conn(0x55e761aa1e00 msgr2=0x55e761aa21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 -- 172.21.15.99:0/3364882010 shutdown_connections 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 -- 172.21.15.99:0/3364882010 wait complete. 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 -- start start 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 Processor -- start 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.721+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 0x55e761ab0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 0x55e761aadc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e7616ee790 con 0x55e761ab0ad0 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e7619c4890 con 0x55e761aa6210 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e761ab3740 con 0x55e761aa58f0 2024-05-14T17:07:32.729 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 10 client.0 _ll_register_callbacks cb 0x55e7619c7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 0x55e761ab0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 0x55e761aadc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 0x55e761ab0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:42004/0 (socket says 172.21.15.99:42004) 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 0x55e761aadc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39428/0 (socket says 172.21.15.99:39428) 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 -- 172.21.15.99:0/2629885499 learned_addr learned my addr 172.21.15.99:0/2629885499 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 -- 172.21.15.99:0/2629885499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 msgr2=0x55e761aad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 -- 172.21.15.99:0/2629885499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 msgr2=0x55e761aadc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 0x55e761aadc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 -- 172.21.15.99:0/2629885499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7617e9620 con 0x55e761aa58f0 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 0x55e761aadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b8affd640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 0x55e761ab0420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0108b0 tx=0x7f0b7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1525356e27d58f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b6eff5640 1 -- 172.21.15.99:0/2629885499 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c02c060 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b6eff5640 1 -- 172.21.15.99:0/2629885499 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f0b7c017030 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b80013120 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b6eff5640 1 -- 172.21.15.99:0/2629885499 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c01f930 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 10 client.25315 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.722+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e761abf280 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b6eff5640 1 -- 172.21.15.99:0/2629885499 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80013120 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b6eff5640 1 -- 172.21.15.99:0/2629885499 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0b7c01b390 con 0x55e761aa58f0 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b93fa8740 10 client.25315 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b93fa8740 10 client.25315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b93fa8740 20 client.25315 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761ac0790 0x55e761ac2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b93fa8740 4 client.25315 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e761a1ab80 con 0x55e761ac0790 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761ac0790 0x55e761ac2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b6eff5640 10 client.25315 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.723+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761ac0790 0x55e761ac2b80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3945f6ad5ba1b53 server_cookie=4e0ceb37ac1abb33 in_seq=0 out_seq=0 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b6eff5640 1 -- 172.21.15.99:0/2629885499 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e761a1ab80 con 0x55e761ac0790 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b6eff5640 10 client.25315 handle_command_reply: tid=0 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 client.25315 shutdown 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 msgr2=0x55e761ab0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 0x55e761ab0420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0108b0 tx=0x7f0b7c004730 comp rx=0 tx=0).stop 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 shutdown_connections 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761ac0790 0x55e761ac2b80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761ab0ad0 0x55e761aadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/2629885499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761aa58f0 0x55e761ab0420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b6eff5640 0 client.25315 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 >> 172.21.15.99:0/2629885499 conn(0x55e761aa1e00 msgr2=0x55e761aa1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 shutdown_connections 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 1 -- 172.21.15.99:0/2629885499 wait complete. 2024-05-14T17:07:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.724+0000 7f0b93fa8740 20 client.25315 trim_cache size 0 max 16384 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 Processor -- start 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 -- start start 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761abc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761aac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1b2f50 con 0x55e761aa58f0 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1b0820 con 0x55e761aa6210 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1ae670 con 0x55e761ab0ad0 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761abc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761abc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39444/0 (socket says 172.21.15.99:39444) 2024-05-14T17:07:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 -- 172.21.15.99:0/326663431 learned_addr learned my addr 172.21.15.99:0/326663431 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761aac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 -- 172.21.15.99:0/326663431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 msgr2=0x55e761aac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761aac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 -- 172.21.15.99:0/326663431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 msgr2=0x55e761aabfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aabfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 -- 172.21.15.99:0/326663431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e761aadfe0 con 0x55e761aa58f0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b8affd640 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761abc640 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c000c00 tx=0x7f0b7c01bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0dd18a9a38a8fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.740+0000 7f0b70ff9640 1 -- 172.21.15.99:0/326663431 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c02c020 con 0x55e761aa58f0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b70ff9640 1 -- 172.21.15.99:0/326663431 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c01b390 con 0x55e761aa58f0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b70ff9640 1 -- 172.21.15.99:0/326663431 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c031770 con 0x55e761aa58f0 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 -- 172.21.15.99:0/326663431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 msgr2=0x55e761abc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761abc640 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c000c00 tx=0x7f0b7c01bf30 comp rx=0 tx=0).stop 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 -- 172.21.15.99:0/326663431 shutdown_connections 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761aac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761aabfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/326663431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761abc640 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 -- 172.21.15.99:0/326663431 >> 172.21.15.99:0/326663431 conn(0x55e761aa1e00 msgr2=0x55e761a9cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 -- 172.21.15.99:0/326663431 shutdown_connections 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 -- 172.21.15.99:0/326663431 wait complete. 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.741+0000 7f0b93fa8740 1 -- start start 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 Processor -- start 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761ab5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761b29020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1aece0 con 0x55e761aa58f0 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1b0150 con 0x55e761aa6210 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1af3b0 con 0x55e761ab0ad0 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b93fa8740 10 client.0 _ll_register_callbacks cb 0x7f0b7c010770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761ab5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761ab5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39450/0 (socket says 172.21.15.99:39450) 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 -- 172.21.15.99:0/445003328 learned_addr learned my addr 172.21.15.99:0/445003328 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761b29020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 -- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 msgr2=0x55e761b29020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761b29020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 -- 172.21.15.99:0/445003328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 msgr2=0x55e761ab5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 -- 172.21.15.99:0/445003328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e761b29560 con 0x55e761aa58f0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8b7fe640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761b29020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b8affd640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761ab5960 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c01bc60 tx=0x7f0b7c01bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a509013e503fcc65 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.742+0000 7f0b6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b6eff5640 1 -- 172.21.15.99:0/445003328 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c03f060 con 0x55e761aa58f0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b6eff5640 1 -- 172.21.15.99:0/445003328 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f0b7c01fda0 con 0x55e761aa58f0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c01b390 con 0x55e761aa58f0 2024-05-14T17:07:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b6eff5640 1 -- 172.21.15.99:0/445003328 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c02c030 con 0x55e761aa58f0 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 10 client.15768 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e761b2a710 con 0x55e761aa58f0 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b6eff5640 1 -- 172.21.15.99:0/445003328 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c01b390 con 0x55e761aa58f0 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b6eff5640 1 -- 172.21.15.99:0/445003328 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0b7c039070 con 0x55e761aa58f0 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 10 client.15768 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 20 client.15768 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761b2ad80 0x55e761b2d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 4 client.15768 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e761a14360 con 0x55e761b2ad80 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.743+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761b2ad80 0x55e761b2d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.744+0000 7f0b6eff5640 10 client.15768 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.744+0000 7f0b8a7fc640 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761b2ad80 0x55e761b2d1a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c59cb69451ff72 server_cookie=c8123600a6e93b5b in_seq=0 out_seq=0 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.824+0000 7f0b6eff5640 1 -- 172.21.15.99:0/445003328 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e761a14360 con 0x55e761b2ad80 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.824+0000 7f0b6eff5640 10 client.15768 handle_command_reply: tid=0 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.824+0000 7f0b93fa8740 1 client.15768 shutdown 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 msgr2=0x55e761ab5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761ab5960 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c01bc60 tx=0x7f0b7c01bfa0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 shutdown_connections 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e761b2ad80 0x55e761b2d1a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e761ab0ad0 0x55e761b29020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e761aa6210 0x55e761ab5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 --2- 172.21.15.99:0/445003328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e761aa58f0 0x55e761ab5960 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b6eff5640 0 client.15768 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 >> 172.21.15.99:0/445003328 conn(0x55e761aa1e00 msgr2=0x55e761aabf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 shutdown_connections 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 1 -- 172.21.15.99:0/445003328 wait complete. 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b93fa8740 20 client.15768 trim_cache size 0 max 16384 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:07:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0b740b2a10 msgr2=0x7f0b740b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0b740b2a10 0x7f0b740b4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55e761aa6e90 tx=0x7f0b7c00b390 comp rx=0 tx=0).stop 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 msgr2=0x7f0b8c0757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 0x7f0b8c0757d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0b84033040 tx=0x7f0b84003970 comp rx=0 tx=0).stop 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 shutdown_connections 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0b740b2a10 0x7f0b740b4ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c108a40 0x7f0b8c0757d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c1033f0 0x7f0b8c075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 --2- 172.21.15.99:0/3946334614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102a30 0x7f0b8c074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.825+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 >> 172.21.15.99:0/3946334614 conn(0x7f0b8c0fe500 msgr2=0x7f0b8c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.826+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 shutdown_connections 2024-05-14T17:07:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:32.826+0000 7f0b9128c640 1 -- 172.21.15.99:0/3946334614 wait complete. 2024-05-14T17:07:33.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:33 smithi150 ceph-mon[36204]: pgmap v120: 97 pgs: 97 active+clean; 47 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-05-14T17:07:33.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:33 smithi099 ceph-mon[31851]: pgmap v120: 97 pgs: 97 active+clean; 47 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-05-14T17:07:33.522 INFO:tasks.workunit.client.0.smithi099.stdout: 1 105211 5.55 MB/sec warmup 105 sec latency 39.555 ms 2024-05-14T17:07:33.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:33 smithi193 ceph-mon[35962]: pgmap v120: 97 pgs: 97 active+clean; 47 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-05-14T17:07:34.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 107070 5.61 MB/sec warmup 106 sec latency 59.763 ms 2024-05-14T17:07:35.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:35 smithi150 ceph-mon[36204]: pgmap v121: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 387 op/s 2024-05-14T17:07:35.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:35 smithi099 ceph-mon[31851]: pgmap v121: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 387 op/s 2024-05-14T17:07:35.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 110070 5.72 MB/sec warmup 107 sec latency 11.940 ms 2024-05-14T17:07:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:35 smithi193 ceph-mon[35962]: pgmap v121: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 387 op/s 2024-05-14T17:07:36.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 112080 5.74 MB/sec warmup 108 sec latency 16.945 ms 2024-05-14T17:07:37.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:37 smithi150 ceph-mon[36204]: pgmap v122: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-14T17:07:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:37 smithi099 ceph-mon[31851]: pgmap v122: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-14T17:07:37.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 114265 5.81 MB/sec warmup 109 sec latency 26.739 ms 2024-05-14T17:07:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:37 smithi193 ceph-mon[35962]: pgmap v122: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-14T17:07:38.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 117088 5.91 MB/sec warmup 110 sec latency 12.121 ms 2024-05-14T17:07:39.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:39 smithi150 ceph-mon[36204]: pgmap v123: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-14T17:07:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:39 smithi099 ceph-mon[31851]: pgmap v123: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-14T17:07:39.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 118525 5.92 MB/sec warmup 111 sec latency 55.564 ms 2024-05-14T17:07:39.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:39 smithi193 ceph-mon[35962]: pgmap v123: 97 pgs: 97 active+clean; 52 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 279 op/s 2024-05-14T17:07:40.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 121082 5.99 MB/sec warmup 112 sec latency 10.024 ms 2024-05-14T17:07:41.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:41 smithi150 ceph-mon[36204]: pgmap v124: 97 pgs: 97 active+clean; 56 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-05-14T17:07:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:41 smithi099 ceph-mon[31851]: pgmap v124: 97 pgs: 97 active+clean; 56 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-05-14T17:07:41.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 123411 6.04 MB/sec warmup 113 sec latency 17.113 ms 2024-05-14T17:07:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:41 smithi193 ceph-mon[35962]: pgmap v124: 97 pgs: 97 active+clean; 56 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 415 op/s 2024-05-14T17:07:42.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 125585 6.10 MB/sec warmup 114 sec latency 17.547 ms 2024-05-14T17:07:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:43 smithi150 ceph-mon[36204]: pgmap v125: 97 pgs: 97 active+clean; 56 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-05-14T17:07:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:43 smithi099 ceph-mon[31851]: pgmap v125: 97 pgs: 97 active+clean; 56 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-05-14T17:07:43.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 128091 6.17 MB/sec warmup 115 sec latency 16.580 ms 2024-05-14T17:07:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:43 smithi193 ceph-mon[35962]: pgmap v125: 97 pgs: 97 active+clean; 56 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-05-14T17:07:44.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 130152 6.18 MB/sec warmup 116 sec latency 73.789 ms 2024-05-14T17:07:45.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:45 smithi150 ceph-mon[36204]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-14T17:07:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:45 smithi099 ceph-mon[31851]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-14T17:07:45.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 132160 6.23 MB/sec warmup 117 sec latency 47.676 ms 2024-05-14T17:07:45.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:45 smithi193 ceph-mon[35962]: pgmap v126: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 -- 172.21.15.99:0/2017847093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 msgr2=0x7fb0c0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/2017847093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c0101e00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009930 tx=0x7fb0b002a420 comp rx=0 tx=0).stop 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 -- 172.21.15.99:0/2017847093 shutdown_connections 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/2017847093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0c0102340 0x7fb0c010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/2017847093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c0101e00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/2017847093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0107940 0x7fb0c0107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.916+0000 7fb0c77c9640 1 -- 172.21.15.99:0/2017847093 >> 172.21.15.99:0/2017847093 conn(0x7fb0c0078ad0 msgr2=0x7fb0c0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 -- 172.21.15.99:0/2017847093 shutdown_connections 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 -- 172.21.15.99:0/2017847093 wait complete. 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 Processor -- start 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 -- start start 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0102340 0x7fb0c01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0c0107940 0x7fb0c019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0111c00 con 0x7fb0c0101980 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0111a80 con 0x7fb0c0107940 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c77c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0111d80 con 0x7fb0c0102340 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c01991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41212/0 (socket says 172.21.15.99:41212) 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.917+0000 7fb0c5565640 1 -- 172.21.15.99:0/1715936466 learned_addr learned my addr 172.21.15.99:0/1715936466 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0102340 0x7fb0c01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0c0107940 0x7fb0c019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5565640 1 -- 172.21.15.99:0/1715936466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0102340 msgr2=0x7fb0c01996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5565640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0102340 0x7fb0c01996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5565640 1 -- 172.21.15.99:0/1715936466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0c0107940 msgr2=0x7fb0c019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5565640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0c0107940 0x7fb0c019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5565640 1 -- 172.21.15.99:0/1715936466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c019e1a0 con 0x7fb0c0101980 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0102340 0x7fb0c01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c5565640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c01991b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e9f0 tx=0x7fb0bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0b67fc640 1 -- 172.21.15.99:0/1715936466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00cd80 con 0x7fb0c0101980 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0b67fc640 1 -- 172.21.15.99:0/1715936466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc0054d0 con 0x7fb0c0101980 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c019e490 con 0x7fb0c0101980 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0b67fc640 1 -- 172.21.15.99:0/1715936466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc010690 con 0x7fb0c0101980 2024-05-14T17:07:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.918+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c01a5d20 con 0x7fb0c0101980 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0b67fc640 1 -- 172.21.15.99:0/1715936466 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb0bc004e90 con 0x7fb0c0101980 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 Processor -- start 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 -- start start 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416da9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562416d1cc30 con 0x562416db4ad0 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562416d0c7d0 con 0x562416daa210 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562416d10810 con 0x562416da98f0 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0c5d66640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416da9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0c4d64640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0c5d66640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416da9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:42928/0 (socket says 172.21.15.99:42928) 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.921+0000 7fb0c5d66640 1 -- 172.21.15.99:0/2157230571 learned_addr learned my addr 172.21.15.99:0/2157230571 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0b67fc640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb0ac0b2a10 0x7fb0ac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5565640 1 -- 172.21.15.99:0/2157230571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 msgr2=0x562416da9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5565640 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416da9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5565640 1 -- 172.21.15.99:0/2157230571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 msgr2=0x562416db6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5565640 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5565640 1 -- 172.21.15.99:0/2157230571 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562416a21210 con 0x562416daa210 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416da9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb0ac0b2a10 0x7fb0ac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0b67fc640 1 -- 172.21.15.99:0/1715936466 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb0bc0e29f0 con 0x7fb0c0101980 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5565640 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db4590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0e0170 tx=0x7fb0bc021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531e10ade93b3827 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb0ac0b2a10 0x7fb0ac0b4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8003df0 tx=0x7fb0b8004a40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0ab7fe640 1 -- 172.21.15.99:0/2157230571 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0a9ad0 con 0x562416daa210 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0ab7fe640 1 -- 172.21.15.99:0/2157230571 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc0a9810 con 0x562416daa210 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.922+0000 7fb0ab7fe640 1 -- 172.21.15.99:0/2157230571 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc005180 con 0x562416daa210 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/2157230571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 msgr2=0x562416db4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db4590 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0e0170 tx=0x7fb0bc021000 comp rx=0 tx=0).stop 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/2157230571 shutdown_connections 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db4590 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/2157230571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416da9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/2157230571 >> 172.21.15.99:0/2157230571 conn(0x562416da5e00 msgr2=0x562416da61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/2157230571 shutdown_connections 2024-05-14T17:07:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/2157230571 wait complete. 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.923+0000 7fb0ca5b6740 1 -- start start 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 Processor -- start 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416db4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624169f2790 con 0x562416db4ad0 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562416cc8890 con 0x562416daa210 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562416db7740 con 0x562416da98f0 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0ca5b6740 10 client.0 _ll_register_callbacks cb 0x562416ccbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c5d66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416db4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:34888/0 (socket says 172.21.15.99:34888) 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c5d66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41220/0 (socket says 172.21.15.99:41220) 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 -- 172.21.15.99:0/248675546 learned_addr learned my addr 172.21.15.99:0/248675546 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 -- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 msgr2=0x562416db4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416db4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 -- 172.21.15.99:0/248675546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 msgr2=0x562416db1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c4d64640 1 -- 172.21.15.99:0/248675546 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562416aed620 con 0x562416daa210 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.924+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db1730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009930 tx=0x7fb0b00317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fccc19f7044cbbd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/248675546 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0035040 con 0x562416daa210 2024-05-14T17:07:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0bc0a9810 con 0x562416daa210 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/248675546 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb0b0004240 con 0x562416daa210 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/248675546 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0b0004440 con 0x562416daa210 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 10 client.25202 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562416dc3160 con 0x562416daa210 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/248675546 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc0a9810 con 0x562416daa210 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/248675546 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0b0004ba0 con 0x562416daa210 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 10 client.25202 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 10 client.25202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 20 client.25202 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416dc4690 0x562416dc6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 4 client.25202 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.925+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562416d1eb80 con 0x562416dc4690 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.926+0000 7fb0c5565640 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416dc4690 0x562416dc6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.926+0000 7fb0a8ff9640 10 client.25202 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.926+0000 7fb0c5565640 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416dc4690 0x562416dc6a80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4005f09e0bd9aff1 server_cookie=58d6bd6110fcc2c3 in_seq=0 out_seq=0 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/248675546 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562416d1eb80 con 0x562416dc4690 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0a8ff9640 10 client.25202 handle_command_reply: tid=0 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 client.25202 shutdown 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 msgr2=0x562416db1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db1730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009930 tx=0x7fb0b00317c0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 shutdown_connections 2024-05-14T17:07:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416dc4690 0x562416dc6a80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416db4ad0 0x562416db1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db1730 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/248675546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416da98f0 0x562416db4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0a8ff9640 0 client.25202 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 >> 172.21.15.99:0/248675546 conn(0x562416da5e00 msgr2=0x562416da5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 shutdown_connections 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/248675546 wait complete. 2024-05-14T17:07:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.927+0000 7fb0ca5b6740 20 client.25202 trim_cache size 0 max 16384 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 Processor -- start 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 -- start start 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416daa210 0x562416dc0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416db4ad0 0x562416db3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b0e20 con 0x562416da98f0 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01ae6f0 con 0x562416db4ad0 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01ac540 con 0x562416daa210 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41226/0 (socket says 172.21.15.99:41226) 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 -- 172.21.15.99:0/503556179 learned_addr learned my addr 172.21.15.99:0/503556179 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416db4ad0 0x562416db3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416daa210 0x562416dc0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 -- 172.21.15.99:0/503556179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416daa210 msgr2=0x562416dc0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416daa210 0x562416dc0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 -- 172.21.15.99:0/503556179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416db4ad0 msgr2=0x562416db3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416db4ad0 0x562416db3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 -- 172.21.15.99:0/503556179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562416da5c30 con 0x562416da98f0 2024-05-14T17:07:45.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416daa210 0x562416dc0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416db4ad0 0x562416db3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.943+0000 7fb0c5565640 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dac530 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e9c0 tx=0x7fb0bc004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c550f751f7f9ce97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0aaffd640 1 -- 172.21.15.99:0/503556179 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0a9470 con 0x562416da98f0 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0aaffd640 1 -- 172.21.15.99:0/503556179 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc0a9810 con 0x562416da98f0 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0aaffd640 1 -- 172.21.15.99:0/503556179 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0a9650 con 0x562416da98f0 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/503556179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 msgr2=0x562416dac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dac530 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e9c0 tx=0x7fb0bc004a90 comp rx=0 tx=0).stop 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/503556179 shutdown_connections 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416db4ad0 0x562416db3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416daa210 0x562416dc0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/503556179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dac530 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/503556179 >> 172.21.15.99:0/503556179 conn(0x562416da5e00 msgr2=0x562416dabb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/503556179 shutdown_connections 2024-05-14T17:07:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.944+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/503556179 wait complete. 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 -- start start 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 Processor -- start 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dbcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416db4ad0 0x562416db84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01acbb0 con 0x562416da98f0 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01ae020 con 0x562416daa210 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01ad280 con 0x562416db4ad0 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0ca5b6740 10 client.0 _ll_register_callbacks cb 0x7fb0b0032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dbcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0c5565640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dbcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41228/0 (socket says 172.21.15.99:41228) 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0c5565640 1 -- 172.21.15.99:0/784656553 learned_addr learned my addr 172.21.15.99:0/784656553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0c5d66640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416db4ad0 0x562416db84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.945+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c5565640 1 -- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416db4ad0 msgr2=0x562416db84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c5565640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416db4ad0 0x562416db84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c5565640 1 -- 172.21.15.99:0/784656553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 msgr2=0x562416db7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c5565640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c5565640 1 -- 172.21.15.99:0/784656553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562416db8a10 con 0x562416da98f0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c5565640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dbcc20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc005790 tx=0x7fb0bc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51f991a467e3c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/784656553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc005180 con 0x562416da98f0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/784656553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb0bc0e61d0 con 0x562416da98f0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0bc0a9810 con 0x562416da98f0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/784656553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0bc0a9ad0 con 0x562416da98f0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 10 client.15786 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562416e2ecd0 con 0x562416da98f0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/784656553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc0a9810 con 0x562416da98f0 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/784656553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0bc0a5f90 con 0x562416da98f0 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 20 client.15786 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416e2f010 0x562416e31400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 4 client.15786 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562416d18360 con 0x562416e2f010 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.946+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416e2f010 0x562416e31400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.947+0000 7fb0a8ff9640 10 client.15786 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:45.947+0000 7fb0c4d64640 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416e2f010 0x562416e31400 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eda8197fb7c084d server_cookie=5a2a95ecdb090098 in_seq=0 out_seq=0 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.002+0000 7fb0a8ff9640 1 -- 172.21.15.99:0/784656553 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562416d18360 con 0x562416e2f010 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.002+0000 7fb0a8ff9640 10 client.15786 handle_command_reply: tid=0 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.002+0000 7fb0ca5b6740 1 client.15786 shutdown 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 msgr2=0x562416dbcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dbcc20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc005790 tx=0x7fb0bc0057c0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 shutdown_connections 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562416e2f010 0x562416e31400 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562416db4ad0 0x562416db84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562416daa210 0x562416db7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0a8ff9640 0 client.15786 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 --2- 172.21.15.99:0/784656553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562416da98f0 0x562416dbcc20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 >> 172.21.15.99:0/784656553 conn(0x562416da5e00 msgr2=0x562416da5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 shutdown_connections 2024-05-14T17:07:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 1 -- 172.21.15.99:0/784656553 wait complete. 2024-05-14T17:07:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.003+0000 7fb0ca5b6740 20 client.15786 trim_cache size 0 max 16384 2024-05-14T17:07:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:07:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:07:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb0ac0b2a10 msgr2=0x7fb0ac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb0ac0b2a10 0x7fb0ac0b4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8003df0 tx=0x7fb0b8004a40 comp rx=0 tx=0).stop 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 msgr2=0x7fb0c01991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c01991b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e9f0 tx=0x7fb0bc00edf0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 shutdown_connections 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb0ac0b2a10 0x7fb0ac0b4ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0c0107940 0x7fb0c019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0c0102340 0x7fb0c01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 --2- 172.21.15.99:0/1715936466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c0101980 0x7fb0c01991b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 >> 172.21.15.99:0/1715936466 conn(0x7fb0c0078ad0 msgr2=0x7fb0c0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 shutdown_connections 2024-05-14T17:07:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:46.004+0000 7fb0c77c9640 1 -- 172.21.15.99:0/1715936466 wait complete. 2024-05-14T17:07:46.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 134927 6.32 MB/sec warmup 118 sec latency 50.690 ms 2024-05-14T17:07:47.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:47 smithi150 ceph-mon[36204]: pgmap v127: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 257 op/s 2024-05-14T17:07:47.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:47 smithi099 ceph-mon[31851]: pgmap v127: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 257 op/s 2024-05-14T17:07:47.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 137167 6.34 MB/sec warmup 119 sec latency 18.552 ms 2024-05-14T17:07:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:47 smithi193 ceph-mon[35962]: pgmap v127: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 257 op/s 2024-05-14T17:07:49.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:49 smithi150 ceph-mon[36204]: pgmap v128: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 257 op/s 2024-05-14T17:07:49.513 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:49 smithi099 ceph-mon[31851]: pgmap v128: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 257 op/s 2024-05-14T17:07:49.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 142022 17.40 MB/sec execute 1 sec latency 36.377 ms 2024-05-14T17:07:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:49 smithi193 ceph-mon[35962]: pgmap v128: 97 pgs: 97 active+clean; 52 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 257 op/s 2024-05-14T17:07:50.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 144212 12.78 MB/sec execute 2 sec latency 12.806 ms 2024-05-14T17:07:51.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:51 smithi150 ceph-mon[36204]: pgmap v129: 97 pgs: 97 active+clean; 47 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 420 op/s 2024-05-14T17:07:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:51 smithi099 ceph-mon[31851]: pgmap v129: 97 pgs: 97 active+clean; 47 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 420 op/s 2024-05-14T17:07:51.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 146182 12.71 MB/sec execute 3 sec latency 31.581 ms 2024-05-14T17:07:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:51 smithi193 ceph-mon[35962]: pgmap v129: 97 pgs: 97 active+clean; 47 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 420 op/s 2024-05-14T17:07:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:52.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:07:52.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:52.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:52.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:52.515 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:07:52.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 148828 13.09 MB/sec execute 4 sec latency 12.068 ms 2024-05-14T17:07:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:07:53.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:53 smithi150 ceph-mon[36204]: pgmap v130: 97 pgs: 97 active+clean; 47 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-05-14T17:07:53.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:53 smithi099 ceph-mon[31851]: pgmap v130: 97 pgs: 97 active+clean; 47 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-05-14T17:07:53.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 151040 12.52 MB/sec execute 5 sec latency 13.654 ms 2024-05-14T17:07:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:53 smithi193 ceph-mon[35962]: pgmap v130: 97 pgs: 97 active+clean; 47 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-05-14T17:07:54.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 153098 12.69 MB/sec execute 6 sec latency 35.149 ms 2024-05-14T17:07:55.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:55 smithi150 ceph-mon[36204]: pgmap v131: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-05-14T17:07:55.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:55 smithi099 ceph-mon[31851]: pgmap v131: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-05-14T17:07:55.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 155758 12.78 MB/sec execute 7 sec latency 15.432 ms 2024-05-14T17:07:55.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:55 smithi193 ceph-mon[35962]: pgmap v131: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-05-14T17:07:56.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 157785 12.56 MB/sec execute 8 sec latency 14.379 ms 2024-05-14T17:07:57.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:57 smithi150 ceph-mon[36204]: pgmap v132: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-14T17:07:57.513 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:57 smithi099 ceph-mon[31851]: pgmap v132: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-14T17:07:57.523 INFO:tasks.workunit.client.0.smithi099.stdout: 1 160205 12.71 MB/sec execute 9 sec latency 16.198 ms 2024-05-14T17:07:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:57 smithi193 ceph-mon[35962]: pgmap v132: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-14T17:07:58.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 162991 12.88 MB/sec execute 10 sec latency 14.066 ms 2024-05-14T17:07:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 -- 172.21.15.99:0/2424707263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 msgr2=0x7fb06010e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 --2- 172.21.15.99:0/2424707263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb06010e800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb058009cd0 tx=0x7fb05802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 -- 172.21.15.99:0/2424707263 shutdown_connections 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 --2- 172.21.15.99:0/2424707263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb06010e800 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 --2- 172.21.15.99:0/2424707263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060101980 0x7fb060101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 --2- 172.21.15.99:0/2424707263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb060107940 0x7fb060107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 -- 172.21.15.99:0/2424707263 >> 172.21.15.99:0/2424707263 conn(0x7fb060078ad0 msgr2=0x7fb060078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 -- 172.21.15.99:0/2424707263 shutdown_connections 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 -- 172.21.15.99:0/2424707263 wait complete. 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 Processor -- start 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.115+0000 7fb0686f8640 1 -- start start 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb060101980 0x7fb060199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb0601996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060107940 0x7fb06019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111bd0 con 0x7fb060102340 2024-05-14T17:07:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111a50 con 0x7fb060107940 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111d50 con 0x7fb060101980 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb0601996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb0601996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51954/0 (socket says 172.21.15.99:51954) 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb066494640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb060101980 0x7fb060199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 -- 172.21.15.99:0/234671990 learned_addr learned my addr 172.21.15.99:0/234671990 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb066c95640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060107940 0x7fb06019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 -- 172.21.15.99:0/234671990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb060101980 msgr2=0x7fb060199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb060101980 0x7fb060199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 -- 172.21.15.99:0/234671990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060107940 msgr2=0x7fb06019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060107940 0x7fb06019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 -- 172.21.15.99:0/234671990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06019e170 con 0x7fb060102340 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb065c93640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb0601996c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e9f0 tx=0x7fb05000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0577fe640 1 -- 172.21.15.99:0/234671990 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05000cd80 con 0x7fb060102340 2024-05-14T17:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0577fe640 1 -- 172.21.15.99:0/234671990 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0500108e0 con 0x7fb060102340 2024-05-14T17:07:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06019e460 con 0x7fb060102340 2024-05-14T17:07:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0577fe640 1 -- 172.21.15.99:0/234671990 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050018980 con 0x7fb060102340 2024-05-14T17:07:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.116+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a5cf0 con 0x7fb060102340 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 Processor -- start 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 -- start start 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 0x55a23dcf7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dd02590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dd02ad0 0x55a23dd04ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a23dc6ac30 con 0x55a23dcf78f0 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a23dc5a7d0 con 0x55a23dcf8210 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb06b40f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a23dc5e810 con 0x55a23dd02ad0 2024-05-14T17:07:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb0577fe640 1 -- 172.21.15.99:0/234671990 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb050010430 con 0x7fb060102340 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb066c95640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 0x55a23dcf7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb066494640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dd02590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb066c95640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 0x55a23dcf7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51958/0 (socket says 172.21.15.99:51958) 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb066c95640 1 -- 172.21.15.99:0/3028363144 learned_addr learned my addr 172.21.15.99:0/3028363144 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.119+0000 7fb065c93640 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dd02ad0 0x55a23dd04ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066494640 1 -- 172.21.15.99:0/3028363144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dd02ad0 msgr2=0x55a23dd04ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066494640 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dd02ad0 0x55a23dd04ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066494640 1 -- 172.21.15.99:0/3028363144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 msgr2=0x55a23dcf7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066494640 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 0x55a23dcf7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066494640 1 -- 172.21.15.99:0/3028363144 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23d96f210 con 0x55a23dcf8210 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066c95640 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 0x55a23dcf7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066494640 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dd02590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005fd0 tx=0x7fb05c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d823c682c905639 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb0577fe640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb04c0b2b10 0x7fb04c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb054ff9640 1 -- 172.21.15.99:0/3028363144 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c01c480 con 0x55a23dcf8210 2024-05-14T17:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb054ff9640 1 -- 172.21.15.99:0/3028363144 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c0118f0 con 0x55a23dcf8210 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb066c95640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb04c0b2b10 0x7fb04c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb054ff9640 1 -- 172.21.15.99:0/3028363144 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c01a7c0 con 0x55a23dcf8210 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.120+0000 7fb0577fe640 1 -- 172.21.15.99:0/234671990 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb0500e28d0 con 0x7fb060102340 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 -- 172.21.15.99:0/3028363144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 msgr2=0x55a23dd02590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb066c95640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb04c0b2b10 0x7fb04c0b4fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb058005e10 tx=0x7fb05803d040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dd02590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005fd0 tx=0x7fb05c0107a0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 -- 172.21.15.99:0/3028363144 shutdown_connections 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dd02ad0 0x55a23dd04ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dd02590 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 --2- 172.21.15.99:0/3028363144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf78f0 0x55a23dcf7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 -- 172.21.15.99:0/3028363144 >> 172.21.15.99:0/3028363144 conn(0x55a23dcf3e00 msgr2=0x55a23dcf41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 -- 172.21.15.99:0/3028363144 shutdown_connections 2024-05-14T17:07:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.121+0000 7fb06b40f740 1 -- 172.21.15.99:0/3028363144 wait complete. 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 -- start start 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 Processor -- start 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 0x55a23dd02390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dcff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dd02ad0 0x55a23dcffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a23d940790 con 0x55a23dd02ad0 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a23dc16890 con 0x55a23dcf8210 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a23dd057a0 con 0x55a23dcf78f0 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb06b40f740 10 client.0 _ll_register_callbacks cb 0x55a23dc19d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb065c93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dcff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 0x55a23dd02390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 0x55a23dd02390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:44704/0 (socket says 172.21.15.99:44704) 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 -- 172.21.15.99:0/4034837973 learned_addr learned my addr 172.21.15.99:0/4034837973 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066c95640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dd02ad0 0x55a23dcffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 -- 172.21.15.99:0/4034837973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 msgr2=0x55a23dcff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dcff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 -- 172.21.15.99:0/4034837973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dd02ad0 msgr2=0x55a23dcffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dd02ad0 0x55a23dcffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066494640 1 -- 172.21.15.99:0/4034837973 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23da3b620 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.122+0000 7fb066c95640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dd02ad0 0x55a23dcffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb066494640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 0x55a23dd02390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb05c010ca0 tx=0x7fb05c00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363ebc344871ff9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb049ffb640 1 -- 172.21.15.99:0/4034837973 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c011aa0 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c0118f0 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb049ffb640 1 -- 172.21.15.99:0/4034837973 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb05c011c40 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb049ffb640 1 -- 172.21.15.99:0/4034837973 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb05c004020 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb06b40f740 10 client.25348 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a23dd111f0 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.123+0000 7fb049ffb640 1 -- 172.21.15.99:0/4034837973 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c0118f0 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb049ffb640 1 -- 172.21.15.99:0/4034837973 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb05c01a500 con 0x55a23dcf78f0 2024-05-14T17:07:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb06b40f740 10 client.25348 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb06b40f740 10 client.25348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb06b40f740 20 client.25348 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb06b40f740 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd12600 0x55a23dd149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb06b40f740 4 client.25348 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a23dc6cb80 con 0x55a23dd12600 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb065c93640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd12600 0x55a23dd149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb065c93640 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd12600 0x55a23dd149f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6a6a99495127d0 server_cookie=9ccffeaa4373492e in_seq=0 out_seq=0 2024-05-14T17:07:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.124+0000 7fb049ffb640 10 client.25348 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb049ffb640 1 -- 172.21.15.99:0/4034837973 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a23dc6cb80 con 0x55a23dd12600 2024-05-14T17:07:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb049ffb640 10 client.25348 handle_command_reply: tid=0 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 client.25348 shutdown 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 msgr2=0x55a23dd02390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 0x55a23dd02390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb05c010ca0 tx=0x7fb05c00b7e0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 shutdown_connections 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd12600 0x55a23dd149f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dd02ad0 0x55a23dcffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf8210 0x55a23dcff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb049ffb640 0 client.25348 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 --2- 172.21.15.99:0/4034837973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf78f0 0x55a23dd02390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 >> 172.21.15.99:0/4034837973 conn(0x55a23dcf3e00 msgr2=0x55a23dcf3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.125+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 shutdown_connections 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.126+0000 7fb06b40f740 1 -- 172.21.15.99:0/4034837973 wait complete. 2024-05-14T17:07:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.126+0000 7fb06b40f740 20 client.25348 trim_cache size 0 max 16384 2024-05-14T17:07:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.141+0000 7fb06b40f740 1 Processor -- start 2024-05-14T17:07:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 -- start start 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd0e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dcf2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd19ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601aed20 con 0x55a23dcf3e00 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ac5f0 con 0x55a23dcf78f0 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb06b40f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601aa440 con 0x55a23dcf8210 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd0e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd0e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51974/0 (socket says 172.21.15.99:51974) 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 -- 172.21.15.99:0/2704411950 learned_addr learned my addr 172.21.15.99:0/2704411950 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066c95640 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd19ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb065c93640 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dcf2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 -- 172.21.15.99:0/2704411950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 msgr2=0x55a23dd19ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd19ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 -- 172.21.15.99:0/2704411950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 msgr2=0x55a23dcf2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dcf2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 -- 172.21.15.99:0/2704411950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23dcf8040 con 0x55a23dcf3e00 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb066494640 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd0e6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb05c011fd0 tx=0x7fb05c00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5581f5083e24 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.142+0000 7fb04bfff640 1 -- 172.21.15.99:0/2704411950 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c010cd0 con 0x55a23dcf3e00 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb04bfff640 1 -- 172.21.15.99:0/2704411950 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c037020 con 0x55a23dcf3e00 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb04bfff640 1 -- 172.21.15.99:0/2704411950 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0041c0 con 0x55a23dcf3e00 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 -- 172.21.15.99:0/2704411950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 msgr2=0x55a23dd0e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd0e6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb05c011fd0 tx=0x7fb05c00b710 comp rx=0 tx=0).stop 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 -- 172.21.15.99:0/2704411950 shutdown_connections 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd19ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dcf2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 --2- 172.21.15.99:0/2704411950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd0e6a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 -- 172.21.15.99:0/2704411950 >> 172.21.15.99:0/2704411950 conn(0x55a23dcffd90 msgr2=0x55a23dcf2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 -- 172.21.15.99:0/2704411950 shutdown_connections 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.143+0000 7fb06b40f740 1 -- 172.21.15.99:0/2704411950 wait complete. 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 -- start start 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 Processor -- start 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dd083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd74be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601aaab0 con 0x55a23dcf3e00 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601abf20 con 0x55a23dcf78f0 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ab180 con 0x55a23dcf8210 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb06b40f740 10 client.0 _ll_register_callbacks cb 0x7fb05c00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd07e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51980/0 (socket says 172.21.15.99:51980) 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 -- 172.21.15.99:0/535525693 learned_addr learned my addr 172.21.15.99:0/535525693 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066c95640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd74be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb065c93640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dd083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 -- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 msgr2=0x55a23dd74be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd74be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 -- 172.21.15.99:0/535525693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 msgr2=0x55a23dd083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dd083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 -- 172.21.15.99:0/535525693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a23dd75220 con 0x55a23dcf3e00 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066c95640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd74be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb066494640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd07e80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb05c000c00 tx=0x7fb05c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd603ba39c59095 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.144+0000 7fb049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb049ffb640 1 -- 172.21.15.99:0/535525693 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c041060 con 0x55a23dcf3e00 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb049ffb640 1 -- 172.21.15.99:0/535525693 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fb05c010ca0 con 0x55a23dcf3e00 2024-05-14T17:07:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c037020 con 0x55a23dcf3e00 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb049ffb640 1 -- 172.21.15.99:0/535525693 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb05c01aaf0 con 0x55a23dcf3e00 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a23dd7c5e0 con 0x55a23dcf3e00 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb049ffb640 1 -- 172.21.15.99:0/535525693 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c037020 con 0x55a23dcf3e00 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb049ffb640 1 -- 172.21.15.99:0/535525693 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb05c004250 con 0x55a23dcf3e00 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 20 client.15804 populate_metadata read hostname 'smithi099' 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd02ad0 0x55a23dd7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 4 client.15804 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a23dc66360 con 0x55a23dd02ad0 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.145+0000 7fb065c93640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd02ad0 0x55a23dd7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.146+0000 7fb049ffb640 10 client.15804 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.146+0000 7fb065c93640 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd02ad0 0x55a23dd7ed20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fdad40a2fdc2e7 server_cookie=704acd9dbcd8c374 in_seq=0 out_seq=0 2024-05-14T17:07:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb049ffb640 1 -- 172.21.15.99:0/535525693 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a23dc66360 con 0x55a23dd02ad0 2024-05-14T17:07:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb049ffb640 10 client.15804 handle_command_reply: tid=0 2024-05-14T17:07:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 client.15804 shutdown 2024-05-14T17:07:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 msgr2=0x55a23dd07e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd07e80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb05c000c00 tx=0x7fb05c0056d0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 shutdown_connections 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a23dd02ad0 0x55a23dd7ed20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb049ffb640 0 client.15804 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a23dcf8210 0x55a23dd74be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a23dcf78f0 0x55a23dd083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 --2- 172.21.15.99:0/535525693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a23dcf3e00 0x55a23dd07e80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 >> 172.21.15.99:0/535525693 conn(0x55a23dcffd90 msgr2=0x55a23dcfdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.206+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 shutdown_connections 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb06b40f740 1 -- 172.21.15.99:0/535525693 wait complete. 2024-05-14T17:07:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb06b40f740 20 client.15804 trim_cache size 0 max 16384 2024-05-14T17:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:07:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb04c0b2b10 msgr2=0x7fb04c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb04c0b2b10 0x7fb04c0b4fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb058005e10 tx=0x7fb05803d040 comp rx=0 tx=0).stop 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 msgr2=0x7fb0601996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb0601996c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e9f0 tx=0x7fb05000eeb0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 shutdown_connections 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb04c0b2b10 0x7fb04c0b4fd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb060107940 0x7fb06019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102340 0x7fb0601996c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 --2- 172.21.15.99:0/234671990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb060101980 0x7fb060199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.207+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 >> 172.21.15.99:0/234671990 conn(0x7fb060078ad0 msgr2=0x7fb06006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.208+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 shutdown_connections 2024-05-14T17:07:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:07:59.208+0000 7fb0686f8640 1 -- 172.21.15.99:0/234671990 wait complete. 2024-05-14T17:07:59.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:07:59 smithi099 ceph-mon[31851]: pgmap v133: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-14T17:07:59.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 164370 12.32 MB/sec execute 11 sec latency 34.303 ms 2024-05-14T17:07:59.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:07:59 smithi193 ceph-mon[35962]: pgmap v133: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-14T17:07:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:07:59 smithi150 ceph-mon[36204]: pgmap v133: 97 pgs: 97 active+clean; 54 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-05-14T17:08:00.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 167099 12.71 MB/sec execute 12 sec latency 12.761 ms 2024-05-14T17:08:01.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:01 smithi099 ceph-mon[31851]: pgmap v134: 97 pgs: 97 active+clean; 59 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 403 op/s 2024-05-14T17:08:01.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 168594 12.15 MB/sec execute 13 sec latency 22.162 ms 2024-05-14T17:08:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:01 smithi193 ceph-mon[35962]: pgmap v134: 97 pgs: 97 active+clean; 59 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 403 op/s 2024-05-14T17:08:01.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:01 smithi150 ceph-mon[36204]: pgmap v134: 97 pgs: 97 active+clean; 59 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 403 op/s 2024-05-14T17:08:02.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 170939 12.27 MB/sec execute 14 sec latency 14.159 ms 2024-05-14T17:08:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:03 smithi099 ceph-mon[31851]: pgmap v135: 97 pgs: 97 active+clean; 59 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-05-14T17:08:03.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 173784 12.61 MB/sec execute 15 sec latency 12.107 ms 2024-05-14T17:08:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:03 smithi193 ceph-mon[35962]: pgmap v135: 97 pgs: 97 active+clean; 59 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-05-14T17:08:03.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:03 smithi150 ceph-mon[36204]: pgmap v135: 97 pgs: 97 active+clean; 59 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-05-14T17:08:04.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 175514 12.26 MB/sec execute 16 sec latency 28.753 ms 2024-05-14T17:08:05.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:05 smithi099 ceph-mon[31851]: pgmap v136: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 460 op/s 2024-05-14T17:08:05.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 177939 12.35 MB/sec execute 17 sec latency 25.382 ms 2024-05-14T17:08:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:05 smithi193 ceph-mon[35962]: pgmap v136: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 460 op/s 2024-05-14T17:08:05.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:05 smithi150 ceph-mon[36204]: pgmap v136: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 460 op/s 2024-05-14T17:08:06.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 180701 12.47 MB/sec execute 18 sec latency 15.439 ms 2024-05-14T17:08:07.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:07 smithi099 ceph-mon[31851]: pgmap v137: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-05-14T17:08:07.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 182607 12.33 MB/sec execute 19 sec latency 19.454 ms 2024-05-14T17:08:07.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:07 smithi193 ceph-mon[35962]: pgmap v137: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-05-14T17:08:07.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:07 smithi150 ceph-mon[36204]: pgmap v137: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-05-14T17:08:08.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 185071 12.41 MB/sec execute 20 sec latency 12.788 ms 2024-05-14T17:08:09.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:09 smithi099 ceph-mon[31851]: pgmap v138: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-05-14T17:08:09.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 187460 12.42 MB/sec execute 21 sec latency 33.209 ms 2024-05-14T17:08:09.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:09 smithi193 ceph-mon[35962]: pgmap v138: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-05-14T17:08:09.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:09 smithi150 ceph-mon[36204]: pgmap v138: 97 pgs: 97 active+clean; 56 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-05-14T17:08:10.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 189477 12.38 MB/sec execute 22 sec latency 14.238 ms 2024-05-14T17:08:11.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:11 smithi099 ceph-mon[31851]: pgmap v139: 97 pgs: 97 active+clean; 62 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 487 op/s 2024-05-14T17:08:11.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 191946 12.45 MB/sec execute 23 sec latency 9.774 ms 2024-05-14T17:08:11.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:11 smithi193 ceph-mon[35962]: pgmap v139: 97 pgs: 97 active+clean; 62 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 487 op/s 2024-05-14T17:08:11.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:11 smithi150 ceph-mon[36204]: pgmap v139: 97 pgs: 97 active+clean; 62 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.7 MiB/s wr, 487 op/s 2024-05-14T17:08:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 -- 172.21.15.99:0/653167602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294102340 msgr2=0x7fa29410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 --2- 172.21.15.99:0/653167602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294102340 0x7fa29410e800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009cd0 tx=0x7fa28c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 -- 172.21.15.99:0/653167602 shutdown_connections 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 --2- 172.21.15.99:0/653167602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294102340 0x7fa29410e800 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 --2- 172.21.15.99:0/653167602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa294101980 0x7fa294101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 --2- 172.21.15.99:0/653167602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa294107940 0x7fa294107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 -- 172.21.15.99:0/653167602 >> 172.21.15.99:0/653167602 conn(0x7fa294078ad0 msgr2=0x7fa294078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.298+0000 7fa299d32640 1 -- 172.21.15.99:0/653167602 shutdown_connections 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 -- 172.21.15.99:0/653167602 wait complete. 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 Processor -- start 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 -- start start 2024-05-14T17:08:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 0x7fa294199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa294102340 0x7fa294199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa294107940 0x7fa29419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa294111c00 con 0x7fa294101980 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa294111a80 con 0x7fa294107940 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa299d32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa294111d80 con 0x7fa294102340 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa2937fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 0x7fa294199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa2937fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 0x7fa294199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39290/0 (socket says 172.21.15.99:39290) 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa2937fe640 1 -- 172.21.15.99:0/1767240105 learned_addr learned my addr 172.21.15.99:0/1767240105 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.299+0000 7fa28bfff640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa294102340 0x7fa294199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa293fff640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa294107940 0x7fa29419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2937fe640 1 -- 172.21.15.99:0/1767240105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa294102340 msgr2=0x7fa294199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2937fe640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa294102340 0x7fa294199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2937fe640 1 -- 172.21.15.99:0/1767240105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa294107940 msgr2=0x7fa29419db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2937fe640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa294107940 0x7fa29419db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2937fe640 1 -- 172.21.15.99:0/1767240105 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29419e230 con 0x7fa294101980 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2937fe640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 0x7fa294199240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e9f0 tx=0x7fa28400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2917fa640 1 -- 172.21.15.99:0/1767240105 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28400cd80 con 0x7fa294101980 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2917fa640 1 -- 172.21.15.99:0/1767240105 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2840054d0 con 0x7fa294101980 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29419e520 con 0x7fa294101980 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa2917fa640 1 -- 172.21.15.99:0/1767240105 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284010690 con 0x7fa294101980 2024-05-14T17:08:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.300+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2941a5db0 con 0x7fa294101980 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.302+0000 7fa29ca50740 1 Processor -- start 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.302+0000 7fa2917fa640 1 -- 172.21.15.99:0/1767240105 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa284004e90 con 0x7fa294101980 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 -- start start 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 0x56386cc83cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc8ead0 0x56386cc90ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56386cbf6c30 con 0x56386cc838f0 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56386cbe67d0 con 0x56386cc8ead0 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa29ca50740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56386cbea810 con 0x56386cc84210 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 0x56386cc83cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 0x56386cc83cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39294/0 (socket says 172.21.15.99:39294) 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 -- 172.21.15.99:0/3087268277 learned_addr learned my addr 172.21.15.99:0/3087268277 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa2937fe640 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa28bfff640 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc8ead0 0x56386cc90ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 -- 172.21.15.99:0/3087268277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 msgr2=0x56386cc8e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 -- 172.21.15.99:0/3087268277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc8ead0 msgr2=0x56386cc90ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc8ead0 0x56386cc90ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 -- 172.21.15.99:0/3087268277 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386c8fb210 con 0x56386cc838f0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa293fff640 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 0x56386cc83cd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009cd0 tx=0x7fa28c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1001c3d387e2361 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.303+0000 7fa2917fa640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2800b2a90 0x7fa2800b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa289ffb640 1 -- 172.21.15.99:0/3087268277 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c049070 con 0x56386cc838f0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa2937fe640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2800b2a90 0x7fa2800b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa289ffb640 1 -- 172.21.15.99:0/3087268277 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c036040 con 0x56386cc838f0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa289ffb640 1 -- 172.21.15.99:0/3087268277 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c004040 con 0x56386cc838f0 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa2917fa640 1 -- 172.21.15.99:0/1767240105 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa2840e28d0 con 0x7fa294101980 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 -- 172.21.15.99:0/3087268277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 msgr2=0x56386cc83cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa2937fe640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2800b2a90 0x7fa2800b4f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2840e0100 tx=0x7fa2840e8550 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 0x56386cc83cd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009cd0 tx=0x7fa28c009a20 comp rx=0 tx=0).stop 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 -- 172.21.15.99:0/3087268277 shutdown_connections 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc8ead0 0x56386cc90ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3087268277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc838f0 0x56386cc83cd0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 -- 172.21.15.99:0/3087268277 >> 172.21.15.99:0/3087268277 conn(0x56386cc7fe00 msgr2=0x56386cc801e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 -- 172.21.15.99:0/3087268277 shutdown_connections 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.304+0000 7fa29ca50740 1 -- 172.21.15.99:0/3087268277 wait complete. 2024-05-14T17:08:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 -- start start 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 Processor -- start 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc8e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc8bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56386c8cc790 con 0x56386cc8ead0 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56386cba2890 con 0x56386cc838f0 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56386cc91740 con 0x56386cc84210 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa29ca50740 10 client.0 _ll_register_callbacks cb 0x56386cba5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa293fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc8bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa293fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc8bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39310/0 (socket says 172.21.15.99:39310) 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa293fff640 1 -- 172.21.15.99:0/3497770444 learned_addr learned my addr 172.21.15.99:0/3497770444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.305+0000 7fa293fff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc8e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa28bfff640 1 -- 172.21.15.99:0/3497770444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 msgr2=0x56386cc8e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa28bfff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc8e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa28bfff640 1 -- 172.21.15.99:0/3497770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 msgr2=0x56386cc8bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa28bfff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc8bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa28bfff640 1 -- 172.21.15.99:0/3497770444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386c9c7620 con 0x56386cc84210 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa293fff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc8bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa28bfff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8b790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x56386cc80f90 tx=0x7fa27c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7cbc9a10940039 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa272ffd640 1 -- 172.21.15.99:0/3497770444 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c004050 con 0x56386cc84210 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c036040 con 0x56386cc84210 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa272ffd640 1 -- 172.21.15.99:0/3497770444 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa27c0041f0 con 0x56386cc84210 2024-05-14T17:08:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa272ffd640 1 -- 172.21.15.99:0/3497770444 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa27c0043f0 con 0x56386cc84210 2024-05-14T17:08:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa29ca50740 10 client.25366 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56386cc9d220 con 0x56386cc84210 2024-05-14T17:08:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa272ffd640 1 -- 172.21.15.99:0/3497770444 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c036040 con 0x56386cc84210 2024-05-14T17:08:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.306+0000 7fa272ffd640 1 -- 172.21.15.99:0/3497770444 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa27c003ae0 con 0x56386cc84210 2024-05-14T17:08:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa29ca50740 10 client.25366 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa29ca50740 10 client.25366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa29ca50740 20 client.25366 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cc9e780 0x56386cca0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa29ca50740 4 client.25366 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56386cbf8b80 con 0x56386cc9e780 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa293fff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cc9e780 0x56386cca0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa272ffd640 10 client.25366 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.307+0000 7fa293fff640 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cc9e780 0x56386cca0b90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ebdd0aa64a6e827 server_cookie=83c90968f7efc613 in_seq=0 out_seq=0 2024-05-14T17:08:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa272ffd640 1 -- 172.21.15.99:0/3497770444 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56386cbf8b80 con 0x56386cc9e780 2024-05-14T17:08:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa272ffd640 10 client.25366 handle_command_reply: tid=0 2024-05-14T17:08:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 client.25366 shutdown 2024-05-14T17:08:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 msgr2=0x56386cc8b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8b790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x56386cc80f90 tx=0x7fa27c012b60 comp rx=0 tx=0).stop 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 shutdown_connections 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cc9e780 0x56386cca0b90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc8bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8b790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3497770444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc8e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa272ffd640 0 client.25366 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 >> 172.21.15.99:0/3497770444 conn(0x56386cc7fe00 msgr2=0x56386cc7f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.308+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 shutdown_connections 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.309+0000 7fa29ca50740 1 -- 172.21.15.99:0/3497770444 wait complete. 2024-05-14T17:08:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.309+0000 7fa29ca50740 20 client.25366 trim_cache size 0 max 16384 2024-05-14T17:08:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 Processor -- start 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 -- start start 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cca64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941b0db0 con 0x56386cc8ead0 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ae680 con 0x56386cc838f0 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.324+0000 7fa29ca50740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ac4d0 con 0x56386cc84210 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cca64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:54550/0 (socket says 172.21.15.99:54550) 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa28bfff640 1 -- 172.21.15.99:0/3796742086 learned_addr learned my addr 172.21.15.99:0/3796742086 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cca64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39326/0 (socket says 172.21.15.99:39326) 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 -- 172.21.15.99:0/3796742086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 msgr2=0x56386cc8dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 -- 172.21.15.99:0/3796742086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 msgr2=0x56386cc9a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc9a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 -- 172.21.15.99:0/3796742086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386cc7b830 con 0x56386cc8ead0 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa293fff640 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cca64a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009620 tx=0x7fa28c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b137ea41570bfe68 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa2897fa640 1 -- 172.21.15.99:0/3796742086 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c049040 con 0x56386cc8ead0 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa2897fa640 1 -- 172.21.15.99:0/3796742086 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c036070 con 0x56386cc8ead0 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.325+0000 7fa2897fa640 1 -- 172.21.15.99:0/3796742086 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0305a0 con 0x56386cc8ead0 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 -- 172.21.15.99:0/3796742086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 msgr2=0x56386cca64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cca64a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009620 tx=0x7fa28c005da0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 -- 172.21.15.99:0/3796742086 shutdown_connections 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cca64a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc84210 0x56386cc8dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 --2- 172.21.15.99:0/3796742086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc838f0 0x56386cc9a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 -- 172.21.15.99:0/3796742086 >> 172.21.15.99:0/3796742086 conn(0x56386cc7fe00 msgr2=0x56386cc86ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 -- 172.21.15.99:0/3796742086 shutdown_connections 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 -- 172.21.15.99:0/3796742086 wait complete. 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 -- start start 2024-05-14T17:08:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.326+0000 7fa29ca50740 1 Processor -- start 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc838f0 0x56386cc926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 0x56386cc92bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc93130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941acb40 con 0x56386cc8ead0 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941adfb0 con 0x56386cc84210 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ad210 con 0x56386cc838f0 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 10 client.0 _ll_register_callbacks cb 0x7fa27c009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc93130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 0x56386cc92bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 0x56386cc92bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:52912/0 (socket says 172.21.15.99:52912) 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa28bfff640 1 -- 172.21.15.99:0/2493338611 learned_addr learned my addr 172.21.15.99:0/2493338611 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc838f0 0x56386cc926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 -- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc838f0 msgr2=0x56386cc926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc838f0 0x56386cc926b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 -- 172.21.15.99:0/2493338611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 msgr2=0x56386cc92bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 0x56386cc92bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 -- 172.21.15.99:0/2493338611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386cd08200 con 0x56386cc8ead0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa28bfff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 0x56386cc92bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa293fff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc93130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa28c006010 tx=0x7fa28c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9cee0dd13058e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa272ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa272ffd640 1 -- 172.21.15.99:0/2493338611 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c05e070 con 0x56386cc8ead0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa272ffd640 1 -- 172.21.15.99:0/2493338611 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa28c0319d0 con 0x56386cc8ead0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.327+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c036070 con 0x56386cc8ead0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa272ffd640 1 -- 172.21.15.99:0/2493338611 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa28c049070 con 0x56386cc8ead0 2024-05-14T17:08:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 10 client.15822 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56386cd094d0 con 0x56386cc8ead0 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa272ffd640 1 -- 172.21.15.99:0/2493338611 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c036070 con 0x56386cc8ead0 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa272ffd640 1 -- 172.21.15.99:0/2493338611 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa28c061030 con 0x56386cc8ead0 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 20 client.15822 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cd09750 0x56386cd0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 4 client.15822 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56386cbf2360 con 0x56386cd09750 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa28bfff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cd09750 0x56386cd0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa272ffd640 10 client.15822 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.328+0000 7fa28bfff640 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cd09750 0x56386cd0bb40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce957e5236ff347 server_cookie=57e14568ada42c18 in_seq=0 out_seq=0 2024-05-14T17:08:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa272ffd640 1 -- 172.21.15.99:0/2493338611 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56386cbf2360 con 0x56386cd09750 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa272ffd640 10 client.15822 handle_command_reply: tid=0 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 client.15822 shutdown 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 msgr2=0x56386cc93130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc93130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa28c006010 tx=0x7fa28c009920 comp rx=0 tx=0).stop 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 shutdown_connections 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56386cd09750 0x56386cd0bb40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa272ffd640 0 client.15822 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56386cc8ead0 0x56386cc93130 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56386cc84210 0x56386cc92bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 --2- 172.21.15.99:0/2493338611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56386cc838f0 0x56386cc926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 >> 172.21.15.99:0/2493338611 conn(0x56386cc7fe00 msgr2=0x56386cc852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 shutdown_connections 2024-05-14T17:08:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 1 -- 172.21.15.99:0/2493338611 wait complete. 2024-05-14T17:08:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.389+0000 7fa29ca50740 20 client.15822 trim_cache size 0 max 16384 2024-05-14T17:08:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:08:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:08:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2800b2a90 msgr2=0x7fa2800b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2800b2a90 0x7fa2800b4f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2840e0100 tx=0x7fa2840e8550 comp rx=0 tx=0).stop 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 msgr2=0x7fa294199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 0x7fa294199240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e9f0 tx=0x7fa28400edf0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 shutdown_connections 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa2800b2a90 0x7fa2800b4f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa294107940 0x7fa29419db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa294102340 0x7fa294199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 --2- 172.21.15.99:0/1767240105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa294101980 0x7fa294199240 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 >> 172.21.15.99:0/1767240105 conn(0x7fa294078ad0 msgr2=0x7fa29406a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 shutdown_connections 2024-05-14T17:08:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:12.390+0000 7fa299d32640 1 -- 172.21.15.99:0/1767240105 wait complete. 2024-05-14T17:08:12.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 194045 12.26 MB/sec execute 24 sec latency 53.379 ms 2024-05-14T17:08:13.504 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:13 smithi150 ceph-mon[36204]: pgmap v140: 97 pgs: 97 active+clean; 62 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-05-14T17:08:13.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:13 smithi099 ceph-mon[31851]: pgmap v140: 97 pgs: 97 active+clean; 62 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-05-14T17:08:13.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 196040 12.25 MB/sec execute 25 sec latency 31.033 ms 2024-05-14T17:08:13.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:13 smithi193 ceph-mon[35962]: pgmap v140: 97 pgs: 97 active+clean; 62 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-05-14T17:08:14.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 198771 12.44 MB/sec execute 26 sec latency 46.366 ms 2024-05-14T17:08:15.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:15 smithi099 ceph-mon[31851]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-05-14T17:08:15.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 200485 12.22 MB/sec execute 27 sec latency 25.161 ms 2024-05-14T17:08:15.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:15 smithi193 ceph-mon[35962]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-05-14T17:08:15.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:15 smithi150 ceph-mon[36204]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-05-14T17:08:16.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 202852 12.28 MB/sec execute 28 sec latency 14.756 ms 2024-05-14T17:08:17.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:17 smithi099 ceph-mon[31851]: pgmap v142: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-05-14T17:08:17.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 205556 12.35 MB/sec execute 29 sec latency 12.823 ms 2024-05-14T17:08:17.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:17 smithi193 ceph-mon[35962]: pgmap v142: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-05-14T17:08:17.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:17 smithi150 ceph-mon[36204]: pgmap v142: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-05-14T17:08:18.524 INFO:tasks.workunit.client.0.smithi099.stdout: 1 207433 12.27 MB/sec execute 30 sec latency 13.524 ms 2024-05-14T17:08:19.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:19 smithi099 ceph-mon[31851]: pgmap v143: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-05-14T17:08:19.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 209898 12.32 MB/sec execute 31 sec latency 37.034 ms 2024-05-14T17:08:19.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:19 smithi193 ceph-mon[35962]: pgmap v143: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-05-14T17:08:19.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:19 smithi150 ceph-mon[36204]: pgmap v143: 97 pgs: 97 active+clean; 60 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 320 op/s 2024-05-14T17:08:20.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 212487 12.35 MB/sec execute 32 sec latency 39.500 ms 2024-05-14T17:08:21.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:21 smithi099 ceph-mon[31851]: pgmap v144: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-05-14T17:08:21.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 214429 12.31 MB/sec execute 33 sec latency 15.894 ms 2024-05-14T17:08:21.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:21 smithi193 ceph-mon[35962]: pgmap v144: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-05-14T17:08:21.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:21 smithi150 ceph-mon[36204]: pgmap v144: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-05-14T17:08:22.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 216886 12.36 MB/sec execute 34 sec latency 63.937 ms 2024-05-14T17:08:23.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:23 smithi099 ceph-mon[31851]: pgmap v145: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 322 op/s 2024-05-14T17:08:23.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 219351 12.37 MB/sec execute 35 sec latency 15.165 ms 2024-05-14T17:08:23.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:23 smithi193 ceph-mon[35962]: pgmap v145: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 322 op/s 2024-05-14T17:08:23.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:23 smithi150 ceph-mon[36204]: pgmap v145: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 322 op/s 2024-05-14T17:08:24.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 221458 12.35 MB/sec execute 36 sec latency 21.252 ms 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 -- 172.21.15.99:0/2000138962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724103420 msgr2=0x7f072410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2000138962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724103420 0x7f072410f930 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f071c009cd0 tx=0x7f071c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 -- 172.21.15.99:0/2000138962 shutdown_connections 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2000138962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724103420 0x7f072410f930 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2000138962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724102a60 0x7f0724102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2000138962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724108a70 0x7f0724108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 -- 172.21.15.99:0/2000138962 >> 172.21.15.99:0/2000138962 conn(0x7f0724079ee0 msgr2=0x7f072407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 -- 172.21.15.99:0/2000138962 shutdown_connections 2024-05-14T17:08:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.489+0000 7f072bd2b640 1 -- 172.21.15.99:0/2000138962 wait complete. 2024-05-14T17:08:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 Processor -- start 2024-05-14T17:08:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 -- start start 2024-05-14T17:08:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 0x7f07241a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724103420 0x7f07241a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 0x7f072419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0724119380 con 0x7f0724108a70 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0724119200 con 0x7f0724102a60 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072bd2b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0724119500 con 0x7f0724103420 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072a2c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 0x7f072419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 0x7f07241a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072a2c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 0x7f072419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54132/0 (socket says 172.21.15.99:54132) 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f072a2c8640 1 -- 172.21.15.99:0/2134556094 learned_addr learned my addr 172.21.15.99:0/2134556094 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.490+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 0x7f07241a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:58576/0 (socket says 172.21.15.99:58576) 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f07292c6640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724103420 0x7f07241a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072a2c8640 1 -- 172.21.15.99:0/2134556094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724103420 msgr2=0x7f07241a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072a2c8640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724103420 0x7f07241a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072a2c8640 1 -- 172.21.15.99:0/2134556094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 msgr2=0x7f07241a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072a2c8640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 0x7f07241a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072a2c8640 1 -- 172.21.15.99:0/2134556094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072419fd80 con 0x7f0724108a70 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f07292c6640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724103420 0x7f07241a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f0729ac7640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 0x7f07241a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072a2c8640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 0x7f072419f600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f071c005e30 tx=0x7f071c031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f071affd640 1 -- 172.21.15.99:0/2134556094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c004040 con 0x7f0724108a70 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07241a0010 con 0x7f0724108a70 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f071affd640 1 -- 172.21.15.99:0/2134556094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f071c004690 con 0x7f0724108a70 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f071affd640 1 -- 172.21.15.99:0/2134556094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c030d90 con 0x7f0724108a70 2024-05-14T17:08:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.491+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241ac2c0 con 0x7f0724108a70 2024-05-14T17:08:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.495+0000 7f072ea46740 1 Processor -- start 2024-05-14T17:08:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.495+0000 7f071affd640 1 -- 172.21.15.99:0/2134556094 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f071c0041e0 con 0x7f0724108a70 2024-05-14T17:08:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.495+0000 7f072ea46740 1 -- start start 2024-05-14T17:08:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f072ea46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fc4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f072ea46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 0x5560b0fcf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f072ea46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f072ea46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560b0f37c30 con 0x5560b0fcfad0 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f072ea46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5560b0f277d0 con 0x5560b0fc48f0 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f072ea46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560b0f2b810 con 0x5560b0fc5210 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f07292c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 0x5560b0fcf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 0x5560b0fcf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:50936/0 (socket says 172.21.15.99:50936) 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f0729ac7640 1 -- 172.21.15.99:0/3057651691 learned_addr learned my addr 172.21.15.99:0/3057651691 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.496+0000 7f07292c6640 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fc4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f0729ac7640 1 -- 172.21.15.99:0/3057651691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 msgr2=0x5560b0fc4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f0729ac7640 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fc4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f0729ac7640 1 -- 172.21.15.99:0/3057651691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 msgr2=0x5560b0fd1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f0729ac7640 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f0729ac7640 1 -- 172.21.15.99:0/3057651691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b0c3c210 con 0x5560b0fc5210 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f0729ac7640 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 0x5560b0fcf590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07200114f0 tx=0x7f07200119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=258ff77c8742e951 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f071affd640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f07100b2a00 0x7f07100b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f070ffff640 1 -- 172.21.15.99:0/3057651691 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072001e070 con 0x5560b0fc5210 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f070ffff640 1 -- 172.21.15.99:0/3057651691 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720011250 con 0x5560b0fc5210 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.497+0000 7f070ffff640 1 -- 172.21.15.99:0/3057651691 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720019870 con 0x5560b0fc5210 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.498+0000 7f071affd640 1 -- 172.21.15.99:0/2134556094 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f071c02e030 con 0x7f0724108a70 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.498+0000 7f07292c6640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f07100b2a00 0x7f07100b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.498+0000 7f072ea46740 1 -- 172.21.15.99:0/3057651691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 msgr2=0x5560b0fcf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.498+0000 7f072ea46740 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 0x5560b0fcf590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07200114f0 tx=0x7f07200119b0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f072ea46740 1 -- 172.21.15.99:0/3057651691 shutdown_connections 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f072ea46740 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f072ea46740 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc5210 0x5560b0fcf590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f072ea46740 1 --2- 172.21.15.99:0/3057651691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fc4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f072ea46740 1 -- 172.21.15.99:0/3057651691 >> 172.21.15.99:0/3057651691 conn(0x5560b0fc0e00 msgr2=0x5560b0fc11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f07292c6640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f07100b2a00 0x7f07100b4ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5560b0fc2220 tx=0x7f07140095d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.499+0000 7f072ea46740 1 -- 172.21.15.99:0/3057651691 shutdown_connections 2024-05-14T17:08:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.500+0000 7f072ea46740 1 -- 172.21.15.99:0/3057651691 wait complete. 2024-05-14T17:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.500+0000 7f072ea46740 1 -- start start 2024-05-14T17:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.500+0000 7f072ea46740 1 Processor -- start 2024-05-14T17:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fcf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fc5210 0x5560b0fcc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fcfad0 0x5560b0fccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fcf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fcf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:58602/0 (socket says 172.21.15.99:58602) 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f0729ac7640 1 -- 172.21.15.99:0/152426923 learned_addr learned my addr 172.21.15.99:0/152426923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f07292c6640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fc5210 0x5560b0fcc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560b0c0d790 con 0x5560b0fc5210 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5560b0ee3890 con 0x5560b0fc48f0 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560b0fd27a0 con 0x5560b0fcfad0 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.501+0000 7f072ea46740 10 client.0 _ll_register_callbacks cb 0x5560b0ee6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f072a2c8640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fcfad0 0x5560b0fccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f0729ac7640 1 -- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fcfad0 msgr2=0x5560b0fccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f0729ac7640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fcfad0 0x5560b0fccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f0729ac7640 1 -- 172.21.15.99:0/152426923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fc5210 msgr2=0x5560b0fcc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f0729ac7640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fc5210 0x5560b0fcc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f0729ac7640 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b0d08620 con 0x5560b0fc48f0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f072a2c8640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fcfad0 0x5560b0fccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f0729ac7640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fcf410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0720000c00 tx=0x7f0720019430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4e68c97a4b2c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f070d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f070d7fa640 1 -- 172.21.15.99:0/152426923 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720029060 con 0x5560b0fc48f0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0720011250 con 0x5560b0fc48f0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f070d7fa640 1 -- 172.21.15.99:0/152426923 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f0720019ce0 con 0x5560b0fc48f0 2024-05-14T17:08:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f070d7fa640 1 -- 172.21.15.99:0/152426923 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f072001e040 con 0x5560b0fc48f0 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f072ea46740 10 client.25256 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.502+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560b0fde1c0 con 0x5560b0fc48f0 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f070d7fa640 1 -- 172.21.15.99:0/152426923 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07200054e0 con 0x5560b0fc48f0 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f070d7fa640 1 -- 172.21.15.99:0/152426923 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0720004bb0 con 0x5560b0fc48f0 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f072ea46740 10 client.25256 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f072ea46740 10 client.25256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f072ea46740 20 client.25256 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f072ea46740 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b0fdf620 0x5560b0fe1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f072ea46740 4 client.25256 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0720011250 con 0x5560b0fdf620 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.503+0000 7f07292c6640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b0fdf620 0x5560b0fe1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.504+0000 7f07292c6640 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b0fdf620 0x5560b0fe1a10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259c17273ef85026 server_cookie=1a7bca80006865d1 in_seq=0 out_seq=0 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.504+0000 7f070d7fa640 10 client.25256 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f070d7fa640 1 -- 172.21.15.99:0/152426923 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0720011250 con 0x5560b0fdf620 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f070d7fa640 10 client.25256 handle_command_reply: tid=0 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 client.25256 shutdown 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 msgr2=0x5560b0fcf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fcf410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0720000c00 tx=0x7f0720019430 comp rx=0 tx=0).stop 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 shutdown_connections 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b0fdf620 0x5560b0fe1a10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fcfad0 0x5560b0fccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fc5210 0x5560b0fcc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 --2- 172.21.15.99:0/152426923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc48f0 0x5560b0fcf410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 >> 172.21.15.99:0/152426923 conn(0x5560b0fc0e00 msgr2=0x5560b0fc0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f070d7fa640 0 client.25256 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 shutdown_connections 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.505+0000 7f072ea46740 1 -- 172.21.15.99:0/152426923 wait complete. 2024-05-14T17:08:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.506+0000 7f072ea46740 20 client.25256 trim_cache size 0 max 16384 2024-05-14T17:08:25.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:25 smithi099 ceph-mon[31851]: pgmap v146: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-05-14T17:08:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.520+0000 7f072ea46740 1 Processor -- start 2024-05-14T17:08:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.520+0000 7f072ea46740 1 -- start start 2024-05-14T17:08:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072ea46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fdb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072ea46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fcc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072ea46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072ea46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07241b73b0 con 0x5560b0fcfad0 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072ea46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07241b4c80 con 0x5560b0fc5210 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072ea46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07241b2ad0 con 0x5560b0fc48f0 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54162/0 (socket says 172.21.15.99:54162) 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 -- 172.21.15.99:0/3669103169 learned_addr learned my addr 172.21.15.99:0/3669103169 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fdb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f07292c6640 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fcc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 -- 172.21.15.99:0/3669103169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 msgr2=0x5560b0fdb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fdb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 -- 172.21.15.99:0/3669103169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 msgr2=0x5560b0fcc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fcc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 -- 172.21.15.99:0/3669103169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b0fc82b0 con 0x5560b0fcfad0 2024-05-14T17:08:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f072a2c8640 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fccbe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f071c0cac80 tx=0x7f071c0ca640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769e684759287d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f070f7fe640 1 -- 172.21.15.99:0/3669103169 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c004840 con 0x5560b0fcfad0 2024-05-14T17:08:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f070f7fe640 1 -- 172.21.15.99:0/3669103169 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f071c036070 con 0x5560b0fcfad0 2024-05-14T17:08:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.521+0000 7f070f7fe640 1 -- 172.21.15.99:0/3669103169 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c0cacf0 con 0x5560b0fcfad0 2024-05-14T17:08:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 -- 172.21.15.99:0/3669103169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 msgr2=0x5560b0fccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fccbe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f071c0cac80 tx=0x7f071c0ca640 comp rx=0 tx=0).stop 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 -- 172.21.15.99:0/3669103169 shutdown_connections 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fccbe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fcc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 --2- 172.21.15.99:0/3669103169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 -- 172.21.15.99:0/3669103169 >> 172.21.15.99:0/3669103169 conn(0x5560b0fc0e00 msgr2=0x5560b0fcbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 -- 172.21.15.99:0/3669103169 shutdown_connections 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 -- 172.21.15.99:0/3669103169 wait complete. 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.522+0000 7f072ea46740 1 -- start start 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 Processor -- start 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fd8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fd5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07241b3140 con 0x5560b0fcfad0 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07241b45b0 con 0x5560b0fc5210 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07241b3810 con 0x5560b0fc48f0 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072ea46740 10 client.0 _ll_register_callbacks cb 0x7f0714009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f0729ac7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fd8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54174/0 (socket says 172.21.15.99:54174) 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 -- 172.21.15.99:0/1814586362 learned_addr learned my addr 172.21.15.99:0/1814586362 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f07292c6640 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fd5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 -- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 msgr2=0x5560b0fd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fd8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 -- 172.21.15.99:0/1814586362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 msgr2=0x5560b0fd5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fd5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 -- 172.21.15.99:0/1814586362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b1048db0 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f072a2c8640 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd5bd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f071c006010 tx=0x7f071c0c71b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74457ec92bf9409c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.523+0000 7f070d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f070d7fa640 1 -- 172.21.15.99:0/1814586362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c004840 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f070d7fa640 1 -- 172.21.15.99:0/1814586362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f071c0c9f50 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071c036070 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f070d7fa640 1 -- 172.21.15.99:0/1814586362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f071c0d3400 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 10 client.15840 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560b1049f80 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f070d7fa640 1 -- 172.21.15.99:0/1814586362 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f071c036070 con 0x5560b0fcfad0 2024-05-14T17:08:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f070d7fa640 1 -- 172.21.15.99:0/1814586362 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f071c04e020 con 0x5560b0fcfad0 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 10 client.15840 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 20 client.15840 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b104a5f0 0x5560b104ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 4 client.15840 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560b0f39b80 con 0x5560b104a5f0 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.524+0000 7f0729ac7640 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b104a5f0 0x5560b104ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.525+0000 7f070d7fa640 10 client.15840 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.525+0000 7f0729ac7640 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b104a5f0 0x5560b104ca10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e936f47f8903acb6 server_cookie=64db5588e42d42a7 in_seq=0 out_seq=0 2024-05-14T17:08:25.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 223873 12.39 MB/sec execute 37 sec latency 45.081 ms 2024-05-14T17:08:25.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:25 smithi193 ceph-mon[35962]: pgmap v146: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-05-14T17:08:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f070d7fa640 1 -- 172.21.15.99:0/1814586362 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560b0f39b80 con 0x5560b104a5f0 2024-05-14T17:08:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f070d7fa640 10 client.15840 handle_command_reply: tid=0 2024-05-14T17:08:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 client.15840 shutdown 2024-05-14T17:08:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 msgr2=0x5560b0fd5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd5bd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f071c006010 tx=0x7f071c0c71b0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 shutdown_connections 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5560b104a5f0 0x5560b104ca10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560b0fcfad0 0x5560b0fd5bd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f070d7fa640 0 client.15840 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5560b0fc5210 0x5560b0fd5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 --2- 172.21.15.99:0/1814586362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b0fc48f0 0x5560b0fd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.587+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 >> 172.21.15.99:0/1814586362 conn(0x5560b0fc0e00 msgr2=0x5560b0fcc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 shutdown_connections 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072ea46740 1 -- 172.21.15.99:0/1814586362 wait complete. 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072ea46740 20 client.15840 trim_cache size 0 max 16384 2024-05-14T17:08:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:08:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:08:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:08:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f07100b2a00 msgr2=0x7f07100b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f07100b2a00 0x7f07100b4ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5560b0fc2220 tx=0x7f07140095d0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 msgr2=0x7f072419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 0x7f072419f600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f071c005e30 tx=0x7f071c031db0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 shutdown_connections 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f07100b2a00 0x7f07100b4ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724108a70 0x7f072419f600 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0724103420 0x7f07241a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 --2- 172.21.15.99:0/2134556094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0724102a60 0x7f07241a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.588+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 >> 172.21.15.99:0/2134556094 conn(0x7f0724079ee0 msgr2=0x7f07241002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.589+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 shutdown_connections 2024-05-14T17:08:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:25.589+0000 7f072bd2b640 1 -- 172.21.15.99:0/2134556094 wait complete. 2024-05-14T17:08:25.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:25 smithi150 ceph-mon[36204]: pgmap v146: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-05-14T17:08:26.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 225909 12.25 MB/sec execute 38 sec latency 91.636 ms 2024-05-14T17:08:27.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:27 smithi099 ceph-mon[31851]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-05-14T17:08:27.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 227930 12.26 MB/sec execute 39 sec latency 16.133 ms 2024-05-14T17:08:27.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:27 smithi193 ceph-mon[35962]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-05-14T17:08:27.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:27 smithi150 ceph-mon[36204]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-05-14T17:08:28.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 230733 12.39 MB/sec execute 40 sec latency 11.561 ms 2024-05-14T17:08:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:29 smithi099 ceph-mon[31851]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-05-14T17:08:29.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 232982 12.29 MB/sec execute 41 sec latency 26.975 ms 2024-05-14T17:08:29.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:29 smithi193 ceph-mon[35962]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-05-14T17:08:29.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:29 smithi150 ceph-mon[36204]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 271 op/s 2024-05-14T17:08:30.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 234814 12.29 MB/sec execute 42 sec latency 30.889 ms 2024-05-14T17:08:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:31 smithi099 ceph-mon[31851]: pgmap v149: 97 pgs: 97 active+clean; 68 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 429 op/s 2024-05-14T17:08:31.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 237498 12.33 MB/sec execute 43 sec latency 52.961 ms 2024-05-14T17:08:31.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:31 smithi193 ceph-mon[35962]: pgmap v149: 97 pgs: 97 active+clean; 68 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 429 op/s 2024-05-14T17:08:31.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:31 smithi150 ceph-mon[36204]: pgmap v149: 97 pgs: 97 active+clean; 68 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 429 op/s 2024-05-14T17:08:32.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 239391 12.28 MB/sec execute 44 sec latency 15.207 ms 2024-05-14T17:08:33.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:33 smithi099 ceph-mon[31851]: pgmap v150: 97 pgs: 97 active+clean; 68 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-14T17:08:33.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 241832 12.32 MB/sec execute 45 sec latency 9.039 ms 2024-05-14T17:08:33.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:33 smithi193 ceph-mon[35962]: pgmap v150: 97 pgs: 97 active+clean; 68 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-14T17:08:33.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:33 smithi150 ceph-mon[36204]: pgmap v150: 97 pgs: 97 active+clean; 68 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-14T17:08:34.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 244589 12.36 MB/sec execute 46 sec latency 21.799 ms 2024-05-14T17:08:35.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:35 smithi099 ceph-mon[31851]: pgmap v151: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 421 op/s 2024-05-14T17:08:35.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 246258 12.31 MB/sec execute 47 sec latency 27.092 ms 2024-05-14T17:08:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:35 smithi193 ceph-mon[35962]: pgmap v151: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 421 op/s 2024-05-14T17:08:35.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:35 smithi150 ceph-mon[36204]: pgmap v151: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 421 op/s 2024-05-14T17:08:36.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 248805 12.34 MB/sec execute 48 sec latency 16.277 ms 2024-05-14T17:08:37.322 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-05-14T17:08:37.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:08:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:37 smithi099 ceph-mon[31851]: pgmap v152: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-14T17:08:37.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 251229 12.33 MB/sec execute 49 sec latency 12.812 ms 2024-05-14T17:08:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:37 smithi193 ceph-mon[35962]: pgmap v152: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-14T17:08:37.578 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:08:37.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:37 smithi150 ceph-mon[36204]: pgmap v152: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-14T17:08:38.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 253367 12.33 MB/sec execute 50 sec latency 13.848 ms 2024-05-14T17:08:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1158709756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 msgr2=0x7fa3c010f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1158709756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 0x7fa3c010f4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009900 tx=0x7fa3b802a510 comp rx=0 tx=0).stop 2024-05-14T17:08:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1158709756 shutdown_connections 2024-05-14T17:08:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1158709756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 0x7fa3c010f4c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1158709756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0104600 0x7fa3c010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1158709756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c0103ce0 0x7fa3c01040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1158709756 >> 172.21.15.99:0/1158709756 conn(0x7fa3c0078ad0 msgr2=0x7fa3c0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1158709756 shutdown_connections 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.679+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1158709756 wait complete. 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 Processor -- start 2024-05-14T17:08:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 -- start start 2024-05-14T17:08:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 0x7fa3c0101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c0104600 0x7fa3c0101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 0x7fa3c01023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0112cd0 con 0x7fa3c010d020 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0112b50 con 0x7fa3c0103ce0 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.680+0000 7fa3c53e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0112e50 con 0x7fa3c0104600 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 0x7fa3c0101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 0x7fa3c0101930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:37318/0 (socket says 172.21.15.99:37318) 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 -- 172.21.15.99:0/1529997136 learned_addr learned my addr 172.21.15.99:0/1529997136 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c0104600 0x7fa3c0101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 0x7fa3c01023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 -- 172.21.15.99:0/1529997136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c0104600 msgr2=0x7fa3c0101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c0104600 0x7fa3c0101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 -- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 msgr2=0x7fa3c01023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 0x7fa3c01023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 -- 172.21.15.99:0/1529997136 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c01a7690 con 0x7fa3c0103ce0 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3beffd640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 0x7fa3c0101930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000eb90 tx=0x7fa3b000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3bcff9640 1 -- 172.21.15.99:0/1529997136 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0004d20 con 0x7fa3c0103ce0 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3bcff9640 1 -- 172.21.15.99:0/1529997136 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b000ee90 con 0x7fa3c0103ce0 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c01a7980 con 0x7fa3c0103ce0 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3bcff9640 1 -- 172.21.15.99:0/1529997136 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0010670 con 0x7fa3c0103ce0 2024-05-14T17:08:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.681+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a7e80 con 0x7fa3c0103ce0 2024-05-14T17:08:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3bcff9640 1 -- 172.21.15.99:0/1529997136 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa3b0005020 con 0x7fa3c0103ce0 2024-05-14T17:08:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 Processor -- start 2024-05-14T17:08:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 -- start start 2024-05-14T17:08:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x555751323cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x555751330ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555751296c30 con 0x5557513238f0 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5557512867d0 con 0x55575132ead0 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.684+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55575128a810 con 0x555751324210 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x555751323cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x555751323cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58822/0 (socket says 172.21.15.99:58822) 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bf7fe640 1 -- 172.21.15.99:0/398256759 learned_addr learned my addr 172.21.15.99:0/398256759 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bcff9640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa3ac0b2a10 0x7fa3ac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x555751330ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa3ac0b2a10 0x7fa3ac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 -- 172.21.15.99:0/398256759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 msgr2=0x555751330ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x555751330ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 -- 172.21.15.99:0/398256759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 msgr2=0x555751323cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x555751323cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 -- 172.21.15.99:0/398256759 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555750f9b210 con 0x555751324210 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x555751330ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bcff9640 1 -- 172.21.15.99:0/1529997136 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa3b00e2c20 con 0x7fa3c0103ce0 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa3ac0b2a10 0x7fa3ac0b4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005c40 tx=0x7fa3b802a510 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3beffd640 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132e590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00e03a0 tx=0x7fa3b001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3398ad5a0d43f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3b57fa640 1 -- 172.21.15.99:0/398256759 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00a9880 con 0x555751324210 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3b57fa640 1 -- 172.21.15.99:0/398256759 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b00e0820 con 0x555751324210 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.685+0000 7fa3b57fa640 1 -- 172.21.15.99:0/398256759 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b004d6c0 con 0x555751324210 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 -- 172.21.15.99:0/398256759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 msgr2=0x55575132e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132e590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00e03a0 tx=0x7fa3b001d000 comp rx=0 tx=0).stop 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 -- 172.21.15.99:0/398256759 shutdown_connections 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x555751330ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132e590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/398256759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x555751323cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 -- 172.21.15.99:0/398256759 >> 172.21.15.99:0/398256759 conn(0x55575131fe00 msgr2=0x5557513201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 -- 172.21.15.99:0/398256759 shutdown_connections 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.686+0000 7fa3c81dc740 1 -- 172.21.15.99:0/398256759 wait complete. 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 -- start start 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 Processor -- start 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x55575132e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x55575132bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555750f6c790 con 0x5557513238f0 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555751242890 con 0x55575132ead0 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555751331740 con 0x555751324210 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3c81dc740 10 client.0 _ll_register_callbacks cb 0x555751245d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x55575132e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x55575132e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58834/0 (socket says 172.21.15.99:58834) 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 -- 172.21.15.99:0/4128576162 learned_addr learned my addr 172.21.15.99:0/4128576162 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x55575132bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 -- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 msgr2=0x55575132b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 -- 172.21.15.99:0/4128576162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 msgr2=0x55575132bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x55575132bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 -- 172.21.15.99:0/4128576162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555751067620 con 0x5557513238f0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3beffd640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x55575132e430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00a9ec0 tx=0x7fa3b00b2cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76beb44e69660c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.687+0000 7fa3a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3a67fc640 1 -- 172.21.15.99:0/4128576162 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00a96a0 con 0x5557513238f0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3a67fc640 1 -- 172.21.15.99:0/4128576162 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa3b00a93a0 con 0x5557513238f0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b00e0820 con 0x5557513238f0 2024-05-14T17:08:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3a67fc640 1 -- 172.21.15.99:0/4128576162 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3b004d6c0 con 0x5557513238f0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 10 client.15852 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55575133d1a0 con 0x5557513238f0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3a67fc640 1 -- 172.21.15.99:0/4128576162 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b00e0820 con 0x5557513238f0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3a67fc640 1 -- 172.21.15.99:0/4128576162 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa3b00a58a0 con 0x5557513238f0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 10 client.15852 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 20 client.15852 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55575133e4f0 0x5557513408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 4 client.15852 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555751298b80 con 0x55575133e4f0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.688+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55575133e4f0 0x5557513408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3b67fc640 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55575133e4f0 0x5557513408e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22982463dad866a server_cookie=adf720bcbf32ab9f in_seq=0 out_seq=0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3a67fc640 10 client.15852 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3a67fc640 1 -- 172.21.15.99:0/4128576162 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555751298b80 con 0x55575133e4f0 2024-05-14T17:08:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3a67fc640 10 client.15852 handle_command_reply: tid=0 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3c81dc740 1 client.15852 shutdown 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 msgr2=0x55575132e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x55575132e430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00a9ec0 tx=0x7fa3b00b2cb0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 shutdown_connections 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.689+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55575133e4f0 0x5557513408e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55575132ead0 0x55575132bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3a67fc640 0 client.15852 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575132b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/4128576162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557513238f0 0x55575132e430 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 >> 172.21.15.99:0/4128576162 conn(0x55575131fe00 msgr2=0x55575131f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 shutdown_connections 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 1 -- 172.21.15.99:0/4128576162 wait complete. 2024-05-14T17:08:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.690+0000 7fa3c81dc740 20 client.15852 trim_cache size 0 max 16384 2024-05-14T17:08:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 Processor -- start 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 -- start start 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x55575133a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575131af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x55575132df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01b3080 con 0x55575132ead0 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01b0950 con 0x5557513238f0 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01ae7a0 con 0x555751324210 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x55575132df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3b67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575131af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x55575133a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3beffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x55575133a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:37346/0 (socket says 172.21.15.99:37346) 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3b67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575131af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:47878/0 (socket says 172.21.15.99:47878) 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x55575132df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58850/0 (socket says 172.21.15.99:58850) 2024-05-14T17:08:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.705+0000 7fa3beffd640 1 -- 172.21.15.99:0/1170605692 learned_addr learned my addr 172.21.15.99:0/1170605692 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3beffd640 1 -- 172.21.15.99:0/1170605692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 msgr2=0x55575131af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3beffd640 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575131af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3beffd640 1 -- 172.21.15.99:0/1170605692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 msgr2=0x55575132df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3beffd640 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x55575132df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3beffd640 1 -- 172.21.15.99:0/1170605692 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575131fc30 con 0x5557513238f0 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3beffd640 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x55575133a610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00a5c70 tx=0x7fa3b00099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da6fec99ea70a311 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3b4ff9640 1 -- 172.21.15.99:0/1170605692 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00a9880 con 0x5557513238f0 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3b4ff9640 1 -- 172.21.15.99:0/1170605692 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b008a070 con 0x5557513238f0 2024-05-14T17:08:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.706+0000 7fa3b4ff9640 1 -- 172.21.15.99:0/1170605692 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00a5df0 con 0x5557513238f0 2024-05-14T17:08:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- 172.21.15.99:0/1170605692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 msgr2=0x55575133a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x55575133a610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00a5c70 tx=0x7fa3b00099d0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- 172.21.15.99:0/1170605692 shutdown_connections 2024-05-14T17:08:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x55575132df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x55575131af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/1170605692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x55575133a610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- 172.21.15.99:0/1170605692 >> 172.21.15.99:0/1170605692 conn(0x55575131fe00 msgr2=0x555751326dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- 172.21.15.99:0/1170605692 shutdown_connections 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- 172.21.15.99:0/1170605692 wait complete. 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- start start 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 Processor -- start 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x555751336cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x555751332070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x5557513325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01aee10 con 0x55575132ead0 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01b0280 con 0x5557513238f0 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.707+0000 7fa3c81dc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c01af4e0 con 0x555751324210 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3c81dc740 10 client.0 _ll_register_callbacks cb 0x7fa3b00a9d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x5557513325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3b67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x555751332070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3b67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x555751332070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:47894/0 (socket says 172.21.15.99:47894) 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3b67fc640 1 -- 172.21.15.99:0/3292882490 learned_addr learned my addr 172.21.15.99:0/3292882490 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3beffd640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x555751336cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 -- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 msgr2=0x555751332070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x555751332070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 -- 172.21.15.99:0/3292882490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 msgr2=0x555751336cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x555751336cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 -- 172.21.15.99:0/3292882490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555751332b70 con 0x55575132ead0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3beffd640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x555751336cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3bf7fe640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x5557513325b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8031440 tx=0x7fa3b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aec746527ebae3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3a67fc640 1 -- 172.21.15.99:0/3292882490 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8035040 con 0x55575132ead0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3a67fc640 1 -- 172.21.15.99:0/3292882490 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fa3b80043e0 con 0x55575132ead0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b008a070 con 0x55575132ead0 2024-05-14T17:08:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3a67fc640 1 -- 172.21.15.99:0/3292882490 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3b805c050 con 0x55575132ead0 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3c81dc740 10 client.15858 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.708+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557513a8840 con 0x55575132ead0 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3a67fc640 1 -- 172.21.15.99:0/3292882490 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b008a070 con 0x55575132ead0 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3a67fc640 1 -- 172.21.15.99:0/3292882490 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa3b8004b80 con 0x55575132ead0 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3c81dc740 10 client.15858 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3c81dc740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3c81dc740 20 client.15858 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5557513a9070 0x5557513ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3c81dc740 4 client.15858 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555751292360 con 0x5557513a9070 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3beffd640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5557513a9070 0x5557513ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3beffd640 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5557513a9070 0x5557513ab490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67e708f3429647d server_cookie=1d8a93f8f3c790ee in_seq=0 out_seq=0 2024-05-14T17:08:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.709+0000 7fa3a67fc640 10 client.15858 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3a67fc640 1 -- 172.21.15.99:0/3292882490 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555751292360 con 0x5557513a9070 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3a67fc640 10 client.15858 handle_command_reply: tid=0 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 client.15858 shutdown 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 msgr2=0x5557513325b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x5557513325b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8031440 tx=0x7fa3b8009260 comp rx=0 tx=0).stop 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 shutdown_connections 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5557513a9070 0x5557513ab490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55575132ead0 0x5557513325b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3a67fc640 0 client.15858 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555751324210 0x555751332070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 --2- 172.21.15.99:0/3292882490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557513238f0 0x555751336cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 >> 172.21.15.99:0/3292882490 conn(0x55575131fe00 msgr2=0x55575131f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 shutdown_connections 2024-05-14T17:08:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 1 -- 172.21.15.99:0/3292882490 wait complete. 2024-05-14T17:08:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.786+0000 7fa3c81dc740 20 client.15858 trim_cache size 0 max 16384 2024-05-14T17:08:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:08:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:08:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa3ac0b2a10 msgr2=0x7fa3ac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa3ac0b2a10 0x7fa3ac0b4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005c40 tx=0x7fa3b802a510 comp rx=0 tx=0).stop 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 msgr2=0x7fa3c0101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 0x7fa3c0101930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000eb90 tx=0x7fa3b000c6a0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 shutdown_connections 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa3ac0b2a10 0x7fa3ac0b4ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c010d020 0x7fa3c01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c0104600 0x7fa3c0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 --2- 172.21.15.99:0/1529997136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3c0103ce0 0x7fa3c0101930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 >> 172.21.15.99:0/1529997136 conn(0x7fa3c0078ad0 msgr2=0x7fa3c00fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 shutdown_connections 2024-05-14T17:08:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:38.787+0000 7fa3c53e4640 1 -- 172.21.15.99:0/1529997136 wait complete. 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/80451245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0108510 msgr2=0x7f2ec01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/80451245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0108510 0x7f2ec01088f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009990 tx=0x7f2eb4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/80451245 shutdown_connections 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/80451245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/80451245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0108510 0x7f2ec01088f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/80451245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec0102430 0x7f2ec01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/80451245 >> 172.21.15.99:0/80451245 conn(0x7f2ec00793b0 msgr2=0x7f2ec00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.443+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/80451245 shutdown_connections 2024-05-14T17:08:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/80451245 wait complete. 2024-05-14T17:08:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 Processor -- start 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 -- start start 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0102430 0x7f2ec019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec0108510 0x7f2ec01a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01125e0 con 0x7f2ec0102430 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0112460 con 0x7f2ec0102df0 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.444+0000 7f2ec6fbd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0112760 con 0x7f2ec0108510 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec019af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:37370/0 (socket says 172.21.15.99:37370) 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 -- 172.21.15.99:0/4039810601 learned_addr learned my addr 172.21.15.99:0/4039810601 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ec4d59700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0102430 0x7f2ec019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ec555a700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec0108510 0x7f2ec01a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 -- 172.21.15.99:0/4039810601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec0108510 msgr2=0x7f2ec01a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec0108510 0x7f2ec01a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 -- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0102430 msgr2=0x7f2ec019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0102430 0x7f2ec019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 -- 172.21.15.99:0/4039810601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec01a5fe0 con 0x7f2ec0102df0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ec4d59700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0102430 0x7f2ec019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebffff700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec019af60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e9c0 tx=0x7f2eac00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00cbd0 con 0x7f2ec0102df0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eac00ee60 con 0x7f2ec0102df0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec0198870 con 0x7f2ec0102df0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.445+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0055c0 con 0x7f2ec0102df0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.446+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec0198d60 con 0x7f2ec0102df0 2024-05-14T17:08:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.446+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec010a9a0 con 0x7f2ec0102df0 2024-05-14T17:08:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.448+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2eac004750 con 0x7f2ec0102df0 2024-05-14T17:08:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.449+0000 7f2ebdffb700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eb00b29f0 0x7f2eb00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.449+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f2eac0e2b40 con 0x7f2ec0102df0 2024-05-14T17:08:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.449+0000 7f2ec4d59700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eb00b29f0 0x7f2eb00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.449+0000 7f2ec4d59700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eb00b29f0 0x7f2eb00b4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b2e0 tx=0x7f2eb401e040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.452+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2eac054df0 con 0x7f2ec0102df0 2024-05-14T17:08:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:39 smithi099 ceph-mon[31851]: pgmap v153: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-14T17:08:39.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 255510 12.34 MB/sec execute 51 sec latency 148.150 ms 2024-05-14T17:08:39.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:39 smithi193 ceph-mon[35962]: pgmap v153: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-14T17:08:39.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:39 smithi150 ceph-mon[36204]: pgmap v153: 97 pgs: 97 active+clean; 75 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-14T17:08:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.744+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec006a230 con 0x7f2ec0102df0 2024-05-14T17:08:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.744+0000 7f2ebdffb700 1 -- 172.21.15.99:0/4039810601 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11876 (secure 0 0 0) 0x7f2eac0aa4f0 con 0x7f2ec0102df0 2024-05-14T17:08:39.744 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:08:39.744 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:06:34.984717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eb00b29f0 msgr2=0x7f2eb00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eb00b29f0 0x7f2eb00b4eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b2e0 tx=0x7f2eb401e040 comp rx=0 tx=0).stop 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 msgr2=0x7f2ec019af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec019af60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e9c0 tx=0x7f2eac00ecd0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 shutdown_connections 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eb00b29f0 0x7f2eb00b4eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec0108510 0x7f2ec01a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec0102430 0x7f2ec019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 --2- 172.21.15.99:0/4039810601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec0102df0 0x7f2ec019af60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 >> 172.21.15.99:0/4039810601 conn(0x7f2ec00793b0 msgr2=0x7f2ec0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 shutdown_connections 2024-05-14T17:08:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:39.747+0000 7f2ec6fbd700 1 -- 172.21.15.99:0/4039810601 wait complete. 2024-05-14T17:08:39.748 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-05-14T17:08:40.437 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:40 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4039810601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:08:40.470 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-05-14T17:08:40.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-05-14T17:08:40.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 257282 12.23 MB/sec execute 52 sec latency 34.770 ms 2024-05-14T17:08:40.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:40 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4039810601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:08:40.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:40 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4039810601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:08:40.741 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:08:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:41 smithi099 ceph-mon[31851]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 412 op/s 2024-05-14T17:08:41.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 259565 12.26 MB/sec execute 53 sec latency 25.827 ms 2024-05-14T17:08:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:41 smithi193 ceph-mon[35962]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 412 op/s 2024-05-14T17:08:41.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:41 smithi150 ceph-mon[36204]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 412 op/s 2024-05-14T17:08:42.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 -- 172.21.15.99:0/3223440348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c106ef0 msgr2=0x7fc03c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 --2- 172.21.15.99:0/3223440348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c106ef0 0x7fc03c10eac0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc03400b0a0 tx=0x7fc03400b3b0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 -- 172.21.15.99:0/3223440348 shutdown_connections 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 --2- 172.21.15.99:0/3223440348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c106ef0 0x7fc03c10eac0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 --2- 172.21.15.99:0/3223440348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c101e10 0x7fc03c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 --2- 172.21.15.99:0/3223440348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c1027c0 0x7fc03c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 -- 172.21.15.99:0/3223440348 >> 172.21.15.99:0/3223440348 conn(0x7fc03c078ae0 msgr2=0x7fc03c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 -- 172.21.15.99:0/3223440348 shutdown_connections 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.420+0000 7fc04094d700 1 -- 172.21.15.99:0/3223440348 wait complete. 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 Processor -- start 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 -- start start 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 0x7fc03c0fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c1027c0 0x7fc03c0fe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c106ef0 0x7fc03c0fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c112660 con 0x7fc03c1027c0 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1124e0 con 0x7fc03c101e10 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc04094d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1127e0 con 0x7fc03c106ef0 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 0x7fc03c0fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.421+0000 7fc039d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c1027c0 0x7fc03c0fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 0x7fc03c0fdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:37380/0 (socket says 172.21.15.99:37380) 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 -- 172.21.15.99:0/3564277746 learned_addr learned my addr 172.21.15.99:0/3564277746 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c106ef0 0x7fc03c0fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 -- 172.21.15.99:0/3564277746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c106ef0 msgr2=0x7fc03c0fe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c106ef0 0x7fc03c0fe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 -- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c1027c0 msgr2=0x7fc03c0fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c1027c0 0x7fc03c0fe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 -- 172.21.15.99:0/3564277746 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c1a8080 con 0x7fc03c101e10 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc039d9b700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c1027c0 0x7fc03c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc03a59c700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 0x7fc03c0fdf20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9c0 tx=0x7fc02c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc02b7fe700 1 -- 172.21.15.99:0/3564277746 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00cbd0 con 0x7fc03c101e10 2024-05-14T17:08:42.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc02b7fe700 1 -- 172.21.15.99:0/3564277746 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc02c00ee60 con 0x7fc03c101e10 2024-05-14T17:08:42.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c1a8370 con 0x7fc03c101e10 2024-05-14T17:08:42.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc02b7fe700 1 -- 172.21.15.99:0/3564277746 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0055c0 con 0x7fc03c101e10 2024-05-14T17:08:42.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.422+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c1a8860 con 0x7fc03c101e10 2024-05-14T17:08:42.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.425+0000 7fc02b7fe700 1 -- 172.21.15.99:0/3564277746 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc02c004750 con 0x7fc03c101e10 2024-05-14T17:08:42.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.426+0000 7fc02b7fe700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc0240b2aa0 0x7fc0240b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.426+0000 7fc039d9b700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc0240b2aa0 0x7fc0240b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.426+0000 7fc02b7fe700 1 -- 172.21.15.99:0/3564277746 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fc02c0e28d0 con 0x7fc03c101e10 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.426+0000 7fc039d9b700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc0240b2aa0 0x7fc0240b4f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc030009680 tx=0x7fc030009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.426+0000 7fc04b1e3b80 1 Processor -- start 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 -- start start 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112eeb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ee5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ee5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564112e43e20 con 0x564112eeb750 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564112e339c0 con 0x564112ee5b60 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564112e37a00 con 0x564112eeae30 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ee5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112eeb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ee5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58898/0 (socket says 172.21.15.99:58898) 2024-05-14T17:08:42.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 -- 172.21.15.99:0/216357003 learned_addr learned my addr 172.21.15.99:0/216357003 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ee5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 -- 172.21.15.99:0/216357003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 msgr2=0x564112eeb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112eeb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 -- 172.21.15.99:0/216357003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 msgr2=0x564112ee5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ee5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 -- 172.21.15.99:0/216357003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564112d83a60 con 0x564112eeb750 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc03a59c700 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112eeb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.427+0000 7fc039d9b700 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ee5540 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e910 tx=0x7fc030014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caee47ce17a28c31 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.428+0000 7fc028ff9700 1 -- 172.21.15.99:0/216357003 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030029070 con 0x564112eeb750 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.428+0000 7fc028ff9700 1 -- 172.21.15.99:0/216357003 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03001c9b0 con 0x564112eeb750 2024-05-14T17:08:42.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.428+0000 7fc028ff9700 1 -- 172.21.15.99:0/216357003 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03001ab00 con 0x564112eeb750 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.428+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/216357003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 msgr2=0x564112ee5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.428+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ee5540 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e910 tx=0x7fc030014000 comp rx=0 tx=0).stop 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/216357003 shutdown_connections 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ee5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112eeb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/216357003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ee5540 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/216357003 >> 172.21.15.99:0/216357003 conn(0x564112e103e0 msgr2=0x564112edfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:42.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/216357003 shutdown_connections 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/216357003 wait complete. 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 -- start start 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.429+0000 7fc04b1e3b80 1 Processor -- start 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 0x564112eee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeb750 0x564112eeedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564112d3a130 con 0x564112eeae30 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564112dd5950 con 0x564112ee5b60 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564112ef2190 con 0x564112eeb750 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 10 client.0 _ll_register_callbacks cb 0x564112dca710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc039d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 0x564112eee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc039d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 0x564112eee870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58912/0 (socket says 172.21.15.99:58912) 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc039d9b700 1 -- 172.21.15.99:0/441010362 learned_addr learned my addr 172.21.15.99:0/441010362 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:42.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeb750 0x564112eeedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 -- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeb750 msgr2=0x564112eeedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeb750 0x564112eeedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 -- 172.21.15.99:0/441010362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 msgr2=0x564112eee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 0x564112eee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 -- 172.21.15.99:0/441010362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564112dd2820 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc039d9b700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 0x564112eee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc03a59c700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef14d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0dfff0 tx=0x7fc02c0aa0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff971cd935757bae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc021ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc021ffb700 1 -- 172.21.15.99:0/441010362 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0a9690 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc021ffb700 1 -- 172.21.15.99:0/441010362 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fc02c0a6170 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.430+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc03001c9b0 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc021ffb700 1 -- 172.21.15.99:0/441010362 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc02c0aa280 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 10 client.25298 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564112efc970 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc021ffb700 1 -- 172.21.15.99:0/441010362 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03001c9b0 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc021ffb700 1 -- 172.21.15.99:0/441010362 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc02c0a54f0 con 0x564112ee5b60 2024-05-14T17:08:42.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 10 client.25298 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 10 client.25298 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 20 client.25298 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112efef00 0x564112f01330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 4 client.25298 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564112eb14f0 con 0x564112efef00 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.431+0000 7fc039d9b700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112efef00 0x564112f01330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.432+0000 7fc039d9b700 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112efef00 0x564112f01330 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b2c88f84521c0f server_cookie=138fe37bc9008b90 in_seq=0 out_seq=0 2024-05-14T17:08:42.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.432+0000 7fc021ffb700 10 client.25298 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc021ffb700 1 -- 172.21.15.99:0/441010362 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564112eb14f0 con 0x564112efef00 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc021ffb700 10 client.25298 handle_command_reply: tid=0 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 client.25298 shutdown 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 msgr2=0x564112ef14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef14d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0dfff0 tx=0x7fc02c0aa0e0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 shutdown_connections 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeb750 0x564112eeedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef14d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112efef00 0x564112f01330 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/441010362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeae30 0x564112eee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc021ffb700 0 client.25298 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 >> 172.21.15.99:0/441010362 conn(0x564112e103e0 msgr2=0x564112eda860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 shutdown_connections 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/441010362 wait complete. 2024-05-14T17:08:42.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.433+0000 7fc04b1e3b80 20 client.25298 trim_cache size 0 max 16384 2024-05-14T17:08:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 Processor -- start 2024-05-14T17:08:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 -- start start 2024-05-14T17:08:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112ee5b60 0x564112efb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112eeae30 0x564112eda970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1b27a0 con 0x564112eeb750 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1b0030 con 0x564112eeae30 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1ade80 con 0x564112ee5b60 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc03ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc03ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58916/0 (socket says 172.21.15.99:58916) 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc03ad9d700 1 -- 172.21.15.99:0/245468159 learned_addr learned my addr 172.21.15.99:0/245468159 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc03a59c700 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112ee5b60 0x564112efb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.453+0000 7fc039d9b700 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112eeae30 0x564112eda970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc03ad9d700 1 -- 172.21.15.99:0/245468159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112ee5b60 msgr2=0x564112efb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112ee5b60 0x564112efb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc03ad9d700 1 -- 172.21.15.99:0/245468159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112eeae30 msgr2=0x564112eda970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112eeae30 0x564112eda970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc03ad9d700 1 -- 172.21.15.99:0/245468159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564112eeb460 con 0x564112eeb750 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef0580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc034015000 tx=0x7fc034009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c2d75948887927 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc023fff700 1 -- 172.21.15.99:0/245468159 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03400bab0 con 0x564112eeb750 2024-05-14T17:08:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc023fff700 1 -- 172.21.15.99:0/245468159 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03400bc50 con 0x564112eeb750 2024-05-14T17:08:42.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.454+0000 7fc023fff700 1 -- 172.21.15.99:0/245468159 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034007c90 con 0x564112eeb750 2024-05-14T17:08:42.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/245468159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 msgr2=0x564112ef0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef0580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc034015000 tx=0x7fc034009520 comp rx=0 tx=0).stop 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/245468159 shutdown_connections 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef0580 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112ee5b60 0x564112efb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/245468159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112eeae30 0x564112eda970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/245468159 >> 172.21.15.99:0/245468159 conn(0x564112e103e0 msgr2=0x564112eefe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/245468159 shutdown_connections 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/245468159 wait complete. 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- start start 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 Processor -- start 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112ef43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1ae4f0 con 0x564112eeb750 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1af960 con 0x564112ee5b60 2024-05-14T17:08:42.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.455+0000 7fc04b1e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1aebc0 con 0x564112eeae30 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc04b1e3b80 10 client.0 _ll_register_callbacks cb 0x564112dca710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58924/0 (socket says 172.21.15.99:58924) 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 -- 172.21.15.99:0/3943176645 learned_addr learned my addr 172.21.15.99:0/3943176645 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc039d9b700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112ef43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03a59c700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 -- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 msgr2=0x564112ef43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112ef43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 -- 172.21.15.99:0/3943176645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 msgr2=0x564112ef7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 -- 172.21.15.99:0/3943176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564112ef4fa0 con 0x564112eeb750 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc03ad9d700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef4930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc0340077d0 tx=0x7fc03400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dbda22682ec7a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc021ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc021ffb700 1 -- 172.21.15.99:0/3943176645 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034006000 con 0x564112eeb750 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc021ffb700 1 -- 172.21.15.99:0/3943176645 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fc034003c10 con 0x564112eeb750 2024-05-14T17:08:42.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc03400bc50 con 0x564112eeb750 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc021ffb700 1 -- 172.21.15.99:0/3943176645 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc034003e30 con 0x564112eeb750 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc04b1e3b80 10 client.15882 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.456+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564112f6a630 con 0x564112eeb750 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc021ffb700 1 -- 172.21.15.99:0/3943176645 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03400bc50 con 0x564112eeb750 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc021ffb700 1 -- 172.21.15.99:0/3943176645 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc034011070 con 0x564112eeb750 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc04b1e3b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc04b1e3b80 10 client.15882 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc04b1e3b80 20 client.15882 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112f6acf0 0x564112f6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc04b1e3b80 4 client.15882 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x564112e45d70 con 0x564112f6acf0 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc03a59c700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112f6acf0 0x564112f6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc021ffb700 10 client.15882 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:42.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.457+0000 7fc03a59c700 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112f6acf0 0x564112f6d110 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d532760875af67 server_cookie=31f00eb246ea4d9f in_seq=0 out_seq=0 2024-05-14T17:08:42.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261863 12.25 MB/sec execute 54 sec latency 55.379 ms 2024-05-14T17:08:42.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc021ffb700 1 -- 172.21.15.99:0/3943176645 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564112e45d70 con 0x564112f6acf0 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc021ffb700 10 client.15882 handle_command_reply: tid=0 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 client.15882 shutdown 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 msgr2=0x564112ef4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef4930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc0340077d0 tx=0x7fc03400be00 comp rx=0 tx=0).stop 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 shutdown_connections 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564112f6acf0 0x564112f6d110 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564112eeb750 0x564112ef4930 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564112ee5b60 0x564112ef7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc021ffb700 0 client.15882 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 --2- 172.21.15.99:0/3943176645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564112eeae30 0x564112ef43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 >> 172.21.15.99:0/3943176645 conn(0x564112e103e0 msgr2=0x564112eee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 shutdown_connections 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 1 -- 172.21.15.99:0/3943176645 wait complete. 2024-05-14T17:08:42.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.527+0000 7fc04b1e3b80 20 client.15882 trim_cache size 0 max 16384 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 27, 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 14, 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "3942b6fd": { 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 14, 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:08:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 146.8, 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:08:42.529 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc0240b2aa0 msgr2=0x7fc0240b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc0240b2aa0 0x7fc0240b4f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc030009680 tx=0x7fc030009260 comp rx=0 tx=0).stop 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 msgr2=0x7fc03c0fdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 0x7fc03c0fdf20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9c0 tx=0x7fc02c00ecd0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 shutdown_connections 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc0240b2aa0 0x7fc0240b4f60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc03c106ef0 0x7fc03c0fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c101e10 0x7fc03c0fdf20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 --2- 172.21.15.99:0/3564277746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03c1027c0 0x7fc03c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:42.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 >> 172.21.15.99:0/3564277746 conn(0x7fc03c078ae0 msgr2=0x7fc03c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:42.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 shutdown_connections 2024-05-14T17:08:42.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:08:42.528+0000 7fc04094d700 1 -- 172.21.15.99:0/3564277746 wait complete. 2024-05-14T17:08:43.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3942b6fd', quiesce took 5.9 seconds. Will release after: 20.9 2024-05-14T17:08:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:43 smithi099 ceph-mon[31851]: pgmap v155: 97 pgs: 97 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 254 op/s 2024-05-14T17:08:43.525 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 12.03 MB/sec execute 55 sec latency 997.656 ms 2024-05-14T17:08:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:43 smithi193 ceph-mon[35962]: pgmap v155: 97 pgs: 97 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 254 op/s 2024-05-14T17:08:43.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:43 smithi150 ceph-mon[36204]: pgmap v155: 97 pgs: 97 active+clean; 81 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 254 op/s 2024-05-14T17:08:44.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 11.81 MB/sec execute 56 sec latency 1997.759 ms 2024-05-14T17:08:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:45 smithi099 ceph-mon[31851]: pgmap v156: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 419 op/s 2024-05-14T17:08:45.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 11.61 MB/sec execute 57 sec latency 2997.812 ms 2024-05-14T17:08:45.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:45 smithi193 ceph-mon[35962]: pgmap v156: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 419 op/s 2024-05-14T17:08:45.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:45 smithi150 ceph-mon[36204]: pgmap v156: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 419 op/s 2024-05-14T17:08:46.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 11.41 MB/sec execute 58 sec latency 3997.916 ms 2024-05-14T17:08:47.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:47 smithi099 ceph-mon[31851]: pgmap v157: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-05-14T17:08:47.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 11.21 MB/sec execute 59 sec latency 4997.988 ms 2024-05-14T17:08:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:47 smithi193 ceph-mon[35962]: pgmap v157: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-05-14T17:08:47.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:47 smithi150 ceph-mon[36204]: pgmap v157: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-05-14T17:08:48.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 11.03 MB/sec execute 60 sec latency 5998.076 ms 2024-05-14T17:08:49.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 10.85 MB/sec execute 61 sec latency 6998.172 ms 2024-05-14T17:08:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:49 smithi193 ceph-mon[35962]: pgmap v158: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-05-14T17:08:49.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:49 smithi150 ceph-mon[36204]: pgmap v158: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-05-14T17:08:49.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:49 smithi099 ceph-mon[31851]: pgmap v158: 97 pgs: 97 active+clean; 73 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-05-14T17:08:50.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 10.67 MB/sec execute 62 sec latency 7998.283 ms 2024-05-14T17:08:51.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 10.50 MB/sec execute 63 sec latency 8998.348 ms 2024-05-14T17:08:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:51 smithi193 ceph-mon[35962]: pgmap v159: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 311 op/s 2024-05-14T17:08:51.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:51 smithi150 ceph-mon[36204]: pgmap v159: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 311 op/s 2024-05-14T17:08:51.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:51 smithi099 ceph-mon[31851]: pgmap v159: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 311 op/s 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 -- 172.21.15.99:0/1072307091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8088107d80 msgr2=0x7f808810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 --2- 172.21.15.99:0/1072307091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8088107d80 0x7f808810f950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8080009ca0 tx=0x7f808002a8c0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 -- 172.21.15.99:0/1072307091 shutdown_connections 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 --2- 172.21.15.99:0/1072307091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8088107d80 0x7f808810f950 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 --2- 172.21.15.99:0/1072307091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088101b70 0x7f80881076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 --2- 172.21.15.99:0/1072307091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80881011c0 0x7f80881015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 -- 172.21.15.99:0/1072307091 >> 172.21.15.99:0/1072307091 conn(0x7f8088077ee0 msgr2=0x7f80880ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 -- 172.21.15.99:0/1072307091 shutdown_connections 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 -- 172.21.15.99:0/1072307091 wait complete. 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 Processor -- start 2024-05-14T17:08:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.885+0000 7f808e447640 1 -- start start 2024-05-14T17:08:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808e447640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 0x7f80881990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808e447640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088107d80 0x7f8088199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808e447640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f808819d980 0x7f808819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808e447640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112ce0 con 0x7f80881011c0 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808e447640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112b60 con 0x7f8088107d80 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808e447640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8088112e60 con 0x7f808819d980 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 0x7f80881990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 0x7f80881990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36152/0 (socket says 172.21.15.99:36152) 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 -- 172.21.15.99:0/973759039 learned_addr learned my addr 172.21.15.99:0/973759039 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f808c9e4640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f808819d980 0x7f808819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f80877fe640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088107d80 0x7f8088199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 -- 172.21.15.99:0/973759039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f808819d980 msgr2=0x7f808819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f808819d980 0x7f808819de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 -- 172.21.15.99:0/973759039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088107d80 msgr2=0x7f8088199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088107d80 0x7f8088199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f8087fff640 1 -- 172.21.15.99:0/973759039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808819e470 con 0x7f80881011c0 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.886+0000 7f80877fe640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088107d80 0x7f8088199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.887+0000 7f8087fff640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 0x7f80881990c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f807800c970 tx=0x7f807800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.887+0000 7f80857fa640 1 -- 172.21.15.99:0/973759039 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078007bb0 con 0x7f80881011c0 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.887+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808819e760 con 0x7f80881011c0 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.887+0000 7f80857fa640 1 -- 172.21.15.99:0/973759039 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8078007d50 con 0x7f80881011c0 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.887+0000 7f80857fa640 1 -- 172.21.15.99:0/973759039 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807800f450 con 0x7f80881011c0 2024-05-14T17:08:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.887+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80881a5f50 con 0x7f80881011c0 2024-05-14T17:08:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.889+0000 7f8091163740 1 Processor -- start 2024-05-14T17:08:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.889+0000 7f8091163740 1 -- start start 2024-05-14T17:08:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.889+0000 7f80857fa640 1 -- 172.21.15.99:0/973759039 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8078004f20 con 0x7f80881011c0 2024-05-14T17:08:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8091163740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 0x5625fb5250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8091163740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb525620 0x5625fb52f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8091163740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb5322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8091163740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5625fb493c30 con 0x5625fb524d00 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8091163740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5625fb4837d0 con 0x5625fb525620 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8091163740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625fb487810 con 0x5625fb52fee0 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 0x5625fb5250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 0x5625fb5250e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36154/0 (socket says 172.21.15.99:36154) 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 -- 172.21.15.99:0/1976900068 learned_addr learned my addr 172.21.15.99:0/1976900068 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f80877fe640 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb5322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8087fff640 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb525620 0x5625fb52f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 -- 172.21.15.99:0/1976900068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 msgr2=0x5625fb5322d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb5322d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 -- 172.21.15.99:0/1976900068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb525620 msgr2=0x5625fb52f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb525620 0x5625fb52f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 -- 172.21.15.99:0/1976900068 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625fb198210 con 0x5625fb524d00 2024-05-14T17:08:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f80857fa640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80700b2a10 0x7f80700b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f808c9e4640 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 0x5625fb5250e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8080005cc0 tx=0x7f8080009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6468f0c2bd7ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.890+0000 7f8087fff640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80700b2a10 0x7f80700b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f806e7fc640 1 -- 172.21.15.99:0/1976900068 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080031df0 con 0x5625fb524d00 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f80857fa640 1 -- 172.21.15.99:0/973759039 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f80780e23c0 con 0x7f80881011c0 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f806e7fc640 1 -- 172.21.15.99:0/1976900068 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8080036040 con 0x5625fb524d00 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f806e7fc640 1 -- 172.21.15.99:0/1976900068 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800042d0 con 0x5625fb524d00 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 -- 172.21.15.99:0/1976900068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 msgr2=0x5625fb5250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8087fff640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80700b2a10 0x7f80700b4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5625fb5262a0 tx=0x7f8078004980 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 0x5625fb5250e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8080005cc0 tx=0x7f8080009930 comp rx=0 tx=0).stop 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 -- 172.21.15.99:0/1976900068 shutdown_connections 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb5322d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb525620 0x5625fb52f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 --2- 172.21.15.99:0/1976900068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb524d00 0x5625fb5250e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 -- 172.21.15.99:0/1976900068 >> 172.21.15.99:0/1976900068 conn(0x5625fb521210 msgr2=0x5625fb5215f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.891+0000 7f8091163740 1 -- 172.21.15.99:0/1976900068 shutdown_connections 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 -- 172.21.15.99:0/1976900068 wait complete. 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 -- start start 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 Processor -- start 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb524d00 0x5625fb52f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb52d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5625fb169790 con 0x5625fb525620 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5625fb43f890 con 0x5625fb524d00 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625fb532b50 con 0x5625fb52fee0 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f8091163740 10 client.0 _ll_register_callbacks cb 0x5625fb442d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.892+0000 7f80877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36164/0 (socket says 172.21.15.99:36164) 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 -- 172.21.15.99:0/4078849478 learned_addr learned my addr 172.21.15.99:0/4078849478 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f808c9e4640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb52d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f808c9e4640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb524d00 0x5625fb52f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 -- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 msgr2=0x5625fb52d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb52d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 -- 172.21.15.99:0/4078849478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb524d00 msgr2=0x5625fb52f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb524d00 0x5625fb52f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 -- 172.21.15.99:0/4078849478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625fb264620 con 0x5625fb525620 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f808c9e4640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb52d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80877fe640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52cb50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f807c011710 tx=0x7f807c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8659c9c148346a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80677fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80677fe640 1 -- 172.21.15.99:0/4078849478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c00f9a0 con 0x5625fb525620 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80677fe640 1 -- 172.21.15.99:0/4078849478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f807c00fb40 con 0x5625fb525620 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8080036040 con 0x5625fb525620 2024-05-14T17:08:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80677fe640 1 -- 172.21.15.99:0/4078849478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f807c00fd40 con 0x5625fb525620 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f8091163740 10 client.15894 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625fb53e6b0 con 0x5625fb525620 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.893+0000 7f80677fe640 1 -- 172.21.15.99:0/4078849478 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8080036040 con 0x5625fb525620 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f80677fe640 1 -- 172.21.15.99:0/4078849478 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f807c018020 con 0x5625fb525620 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f8091163740 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f8091163740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f8091163740 20 client.15894 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f8091163740 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb53fbc0 0x5625fb541fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f8091163740 4 client.15894 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625fb495b80 con 0x5625fb53fbc0 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f808c9e4640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb53fbc0 0x5625fb541fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f808c9e4640 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb53fbc0 0x5625fb541fb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a9bda72722309 server_cookie=d66943cbd8226964 in_seq=0 out_seq=0 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.894+0000 7f80677fe640 10 client.15894 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.895+0000 7f80677fe640 1 -- 172.21.15.99:0/4078849478 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625fb495b80 con 0x5625fb53fbc0 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.895+0000 7f80677fe640 10 client.15894 handle_command_reply: tid=0 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.895+0000 7f8091163740 1 client.15894 shutdown 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 msgr2=0x5625fb52cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52cb50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f807c011710 tx=0x7f807c011bd0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 shutdown_connections 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb53fbc0 0x5625fb541fb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb52fee0 0x5625fb52d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52cb50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f80677fe640 0 client.15894 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 --2- 172.21.15.99:0/4078849478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb524d00 0x5625fb52f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 >> 172.21.15.99:0/4078849478 conn(0x5625fb521210 msgr2=0x5625fb526700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 shutdown_connections 2024-05-14T17:08:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 1 -- 172.21.15.99:0/4078849478 wait complete. 2024-05-14T17:08:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.896+0000 7f8091163740 20 client.15894 trim_cache size 0 max 16384 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 Processor -- start 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 -- start start 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb51c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb526dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb52f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80881aeb70 con 0x5625fb525620 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80881ac440 con 0x5625fb52fee0 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f8091163740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f80881aa290 con 0x5625fb524d00 2024-05-14T17:08:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb526dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb526dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36170/0 (socket says 172.21.15.99:36170) 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 -- 172.21.15.99:0/3597364248 learned_addr learned my addr 172.21.15.99:0/3597364248 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f808c9e4640 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb52f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f808c9e4640 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb51c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 -- 172.21.15.99:0/3597364248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 msgr2=0x5625fb51c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb51c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 -- 172.21.15.99:0/3597364248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 msgr2=0x5625fb52f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb52f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 -- 172.21.15.99:0/3597364248 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625fb525450 con 0x5625fb525620 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f808c9e4640 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb52f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f80877fe640 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb526dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f807c023670 tx=0x7f807c0234d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fccc7f6fb85d66 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.911+0000 7f806dffb640 1 -- 172.21.15.99:0/3597364248 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c029070 con 0x5625fb525620 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f806dffb640 1 -- 172.21.15.99:0/3597364248 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c023860 con 0x5625fb525620 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f806dffb640 1 -- 172.21.15.99:0/3597364248 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c003d90 con 0x5625fb525620 2024-05-14T17:08:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 -- 172.21.15.99:0/3597364248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 msgr2=0x5625fb526dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb526dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f807c023670 tx=0x7f807c0234d0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 -- 172.21.15.99:0/3597364248 shutdown_connections 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb52f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb526dc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 --2- 172.21.15.99:0/3597364248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb51c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 -- 172.21.15.99:0/3597364248 >> 172.21.15.99:0/3597364248 conn(0x5625fb521210 msgr2=0x5625fb5266b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 -- 172.21.15.99:0/3597364248 shutdown_connections 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 -- 172.21.15.99:0/3597364248 wait complete. 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.912+0000 7f8091163740 1 -- start start 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 Processor -- start 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb529d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb5a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80881aa900 con 0x5625fb525620 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80881abd70 con 0x5625fb52fee0 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f80881aafd0 con 0x5625fb524d00 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f8091163740 10 client.0 _ll_register_callbacks cb 0x7f807c008ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36186/0 (socket says 172.21.15.99:36186) 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 -- 172.21.15.99:0/3378524925 learned_addr learned my addr 172.21.15.99:0/3378524925 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f808c9e4640 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb529d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 -- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 msgr2=0x5625fb529d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb529d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 -- 172.21.15.99:0/3378524925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 msgr2=0x5625fb5a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb5a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.913+0000 7f80877fe640 1 -- 172.21.15.99:0/3378524925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625fb5a99b0 con 0x5625fb525620 2024-05-14T17:08:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80877fe640 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52a290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f807c011480 tx=0x7f807c00f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06a49cf4280f492 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80677fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80677fe640 1 -- 172.21.15.99:0/3378524925 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c029040 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80677fe640 1 -- 172.21.15.99:0/3378524925 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f807c005370 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807c023860 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80677fe640 1 -- 172.21.15.99:0/3378524925 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f807c03f050 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625fb5aaa90 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80677fe640 1 -- 172.21.15.99:0/3378524925 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c023860 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f80677fe640 1 -- 172.21.15.99:0/3378524925 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f807c018070 con 0x5625fb525620 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 20 client.15900 populate_metadata read hostname 'smithi099' 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb5ab1a0 0x5625fb5ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 4 client.15900 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.914+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625fb48f360 con 0x5625fb5ab1a0 2024-05-14T17:08:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.915+0000 7f808c9e4640 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb5ab1a0 0x5625fb5ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.915+0000 7f808c9e4640 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb5ab1a0 0x5625fb5ad5c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ccd0c6d20b8c1b server_cookie=7534870b424bac4b in_seq=0 out_seq=0 2024-05-14T17:08:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.915+0000 7f80677fe640 10 client.15900 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.960+0000 7f80677fe640 1 -- 172.21.15.99:0/3378524925 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5625fb48f360 con 0x5625fb5ab1a0 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f80677fe640 10 client.15900 handle_command_reply: tid=0 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 client.15900 shutdown 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 msgr2=0x5625fb52a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52a290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f807c011480 tx=0x7f807c00f8d0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 shutdown_connections 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5625fb5ab1a0 0x5625fb5ad5c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5625fb52fee0 0x5625fb5a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fb525620 0x5625fb52a290 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 --2- 172.21.15.99:0/3378524925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625fb524d00 0x5625fb529d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f80677fe640 0 client.15900 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:08:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 >> 172.21.15.99:0/3378524925 conn(0x5625fb521210 msgr2=0x5625fb51bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 shutdown_connections 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 1 -- 172.21.15.99:0/3378524925 wait complete. 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.961+0000 7f8091163740 20 client.15900 trim_cache size 0 max 16384 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:08:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80700b2a10 msgr2=0x7f80700b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80700b2a10 0x7f80700b4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5625fb5262a0 tx=0x7f8078004980 comp rx=0 tx=0).stop 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 msgr2=0x7f80881990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 0x7f80881990c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f807800c970 tx=0x7f807800ce30 comp rx=0 tx=0).stop 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 shutdown_connections 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80700b2a10 0x7f80700b4ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f808819d980 0x7f808819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8088107d80 0x7f8088199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 --2- 172.21.15.99:0/973759039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80881011c0 0x7f80881990c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 >> 172.21.15.99:0/973759039 conn(0x7f8088077ee0 msgr2=0x7f8088105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 shutdown_connections 2024-05-14T17:08:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:08:51.962+0000 7f808e447640 1 -- 172.21.15.99:0/973759039 wait complete. 2024-05-14T17:08:52.526 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 10.34 MB/sec execute 64 sec latency 9998.391 ms 2024-05-14T17:08:52.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:52 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:52.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:52 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:52.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:52 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:53.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 10.18 MB/sec execute 65 sec latency 10998.472 ms 2024-05-14T17:08:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:53 smithi193 ceph-mon[35962]: pgmap v160: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-14T17:08:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:08:53.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:53 smithi150 ceph-mon[36204]: pgmap v160: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-14T17:08:53.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:53.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:53.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:08:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:53 smithi099 ceph-mon[31851]: pgmap v160: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-05-14T17:08:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:08:54.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 10.02 MB/sec execute 66 sec latency 11998.540 ms 2024-05-14T17:08:55.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.88 MB/sec execute 67 sec latency 12998.593 ms 2024-05-14T17:08:55.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:55 smithi193 ceph-mon[35962]: pgmap v161: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-05-14T17:08:55.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:55 smithi150 ceph-mon[36204]: pgmap v161: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-05-14T17:08:55.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:55 smithi099 ceph-mon[31851]: pgmap v161: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-05-14T17:08:56.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.73 MB/sec execute 68 sec latency 13998.722 ms 2024-05-14T17:08:56.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:56 smithi150 ceph-mon[36204]: pgmap v162: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-14T17:08:56.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:56 smithi099 ceph-mon[31851]: pgmap v162: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-14T17:08:56.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:56 smithi193 ceph-mon[35962]: pgmap v162: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-14T17:08:57.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.59 MB/sec execute 69 sec latency 14998.809 ms 2024-05-14T17:08:58.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.45 MB/sec execute 70 sec latency 15998.910 ms 2024-05-14T17:08:59.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:08:58 smithi099 ceph-mon[31851]: pgmap v163: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-14T17:08:59.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:08:58 smithi193 ceph-mon[35962]: pgmap v163: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-14T17:08:59.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:08:58 smithi150 ceph-mon[36204]: pgmap v163: 97 pgs: 97 active+clean; 70 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-14T17:08:59.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.32 MB/sec execute 71 sec latency 16998.984 ms 2024-05-14T17:09:00.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.19 MB/sec execute 72 sec latency 17999.049 ms 2024-05-14T17:09:01.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:00 smithi099 ceph-mon[31851]: pgmap v164: 97 pgs: 97 active+clean; 75 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 101 op/s 2024-05-14T17:09:01.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:00 smithi193 ceph-mon[35962]: pgmap v164: 97 pgs: 97 active+clean; 75 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 101 op/s 2024-05-14T17:09:01.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:00 smithi150 ceph-mon[36204]: pgmap v164: 97 pgs: 97 active+clean; 75 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 101 op/s 2024-05-14T17:09:01.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 9.06 MB/sec execute 73 sec latency 18999.089 ms 2024-05-14T17:09:02.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.94 MB/sec execute 74 sec latency 19999.126 ms 2024-05-14T17:09:03.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:02 smithi099 ceph-mon[31851]: pgmap v165: 97 pgs: 97 active+clean; 75 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-05-14T17:09:03.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:02 smithi193 ceph-mon[35962]: pgmap v165: 97 pgs: 97 active+clean; 75 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-05-14T17:09:03.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:02 smithi150 ceph-mon[36204]: pgmap v165: 97 pgs: 97 active+clean; 75 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-05-14T17:09:03.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.82 MB/sec execute 75 sec latency 20999.240 ms 2024-05-14T17:09:04.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:09:04.337 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:09:04.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.71 MB/sec execute 76 sec latency 21999.314 ms 2024-05-14T17:09:05.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:04 smithi099 ceph-mon[31851]: pgmap v166: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.5 MiB/s wr, 131 op/s 2024-05-14T17:09:05.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:04 smithi193 ceph-mon[35962]: pgmap v166: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.5 MiB/s wr, 131 op/s 2024-05-14T17:09:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 -- 172.21.15.99:0/291701911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898108a70 msgr2=0x7f5898108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 --2- 172.21.15.99:0/291701911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898108a70 0x7f5898108e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5894009930 tx=0x7f589402a420 comp rx=0 tx=0).stop 2024-05-14T17:09:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 -- 172.21.15.99:0/291701911 shutdown_connections 2024-05-14T17:09:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 --2- 172.21.15.99:0/291701911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5898103420 0x7f589810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 --2- 172.21.15.99:0/291701911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5898102a60 0x7f5898102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 --2- 172.21.15.99:0/291701911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898108a70 0x7f5898108e50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 -- 172.21.15.99:0/291701911 >> 172.21.15.99:0/291701911 conn(0x7f5898079ee0 msgr2=0x7f589807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 -- 172.21.15.99:0/291701911 shutdown_connections 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 -- 172.21.15.99:0/291701911 wait complete. 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 Processor -- start 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 -- start start 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 0x7f5898199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.052+0000 7f58a0304640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5898103420 0x7f5898199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f58a0304640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5898108a70 0x7f589819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f58a0304640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5898112d30 con 0x7f5898102a60 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f58a0304640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5898112bb0 con 0x7f5898108a70 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f58a0304640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5898112eb0 con 0x7f5898103420 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 0x7f5898199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 0x7f5898199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50862/0 (socket says 172.21.15.99:50862) 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 -- 172.21.15.99:0/4058762759 learned_addr learned my addr 172.21.15.99:0/4058762759 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589d89f640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5898103420 0x7f5898199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e8a1640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5898108a70 0x7f589819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 -- 172.21.15.99:0/4058762759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5898103420 msgr2=0x7f5898199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5898103420 0x7f5898199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 -- 172.21.15.99:0/4058762759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5898108a70 msgr2=0x7f589819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5898108a70 0x7f589819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 -- 172.21.15.99:0/4058762759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589819e200 con 0x7f5898102a60 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f589e0a0640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 0x7f5898199210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5894004770 tx=0x7f58940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f588f7fe640 1 -- 172.21.15.99:0/4058762759 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589403f070 con 0x7f5898102a60 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f588f7fe640 1 -- 172.21.15.99:0/4058762759 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f589403b460 con 0x7f5898102a60 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589819e490 con 0x7f5898102a60 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f588f7fe640 1 -- 172.21.15.99:0/4058762759 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894035070 con 0x7f5898102a60 2024-05-14T17:09:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.053+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58981a5d80 con 0x7f5898102a60 2024-05-14T17:09:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.056+0000 7f588f7fe640 1 -- 172.21.15.99:0/4058762759 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f589403b8d0 con 0x7f5898102a60 2024-05-14T17:09:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.056+0000 7f58a3016740 1 Processor -- start 2024-05-14T17:09:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 -- start start 2024-05-14T17:09:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 0x56337e3d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56337e338c30 con 0x56337e3c6210 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56337e3287d0 con 0x56337e3d0ad0 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f58a3016740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56337e32c810 con 0x56337e3c58f0 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 0x56337e3d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 0x56337e3d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50878/0 (socket says 172.21.15.99:50878) 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 -- 172.21.15.99:0/3492651307 learned_addr learned my addr 172.21.15.99:0/3492651307 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589d89f640 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e8a1640 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f588f7fe640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58840b2a10 0x7f58840b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 -- 172.21.15.99:0/3492651307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 msgr2=0x56337e3c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 -- 172.21.15.99:0/3492651307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 msgr2=0x56337e3d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3d2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e0a0640 1 -- 172.21.15.99:0/3492651307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337e03d210 con 0x56337e3c6210 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.057+0000 7f589e8a1640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58840b2a10 0x7f58840b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f588f7fe640 1 -- 172.21.15.99:0/4058762759 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f5894102960 con 0x7f5898102a60 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f589e0a0640 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 0x56337e3d0590 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5894100050 tx=0x7f5894049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60c610af53f761b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f589e8a1640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58840b2a10 0x7f58840b4ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f589000a5a0 tx=0x7f589000c040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f588cbfe640 1 -- 172.21.15.99:0/3492651307 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940c95c0 con 0x56337e3c6210 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f588cbfe640 1 -- 172.21.15.99:0/3492651307 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58940c9780 con 0x56337e3c6210 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f588cbfe640 1 -- 172.21.15.99:0/3492651307 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940044e0 con 0x56337e3c6210 2024-05-14T17:09:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f58a3016740 1 -- 172.21.15.99:0/3492651307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 msgr2=0x56337e3d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.058+0000 7f58a3016740 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 0x56337e3d0590 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5894100050 tx=0x7f5894049000 comp rx=0 tx=0).stop 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- 172.21.15.99:0/3492651307 shutdown_connections 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c6210 0x56337e3d0590 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 --2- 172.21.15.99:0/3492651307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- 172.21.15.99:0/3492651307 >> 172.21.15.99:0/3492651307 conn(0x56337e3c1e00 msgr2=0x56337e3c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- 172.21.15.99:0/3492651307 shutdown_connections 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- 172.21.15.99:0/3492651307 wait complete. 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- start start 2024-05-14T17:09:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 Processor -- start 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c58f0 0x56337e3d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e3cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56337e00e790 con 0x56337e3d0ad0 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56337e2e4890 con 0x56337e3c58f0 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56337e3d34c0 con 0x56337e3c6210 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f58a3016740 10 client.0 _ll_register_callbacks cb 0x56337e2e7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f589e8a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e3cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f589e8a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e3cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50880/0 (socket says 172.21.15.99:50880) 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f589e8a1640 1 -- 172.21.15.99:0/2606943189 learned_addr learned my addr 172.21.15.99:0/2606943189 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.059+0000 7f589e0a0640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c58f0 0x56337e3d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589d89f640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e8a1640 1 -- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 msgr2=0x56337e3cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e8a1640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e8a1640 1 -- 172.21.15.99:0/2606943189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c58f0 msgr2=0x56337e3d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e8a1640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c58f0 0x56337e3d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e8a1640 1 -- 172.21.15.99:0/2606943189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337e109620 con 0x56337e3d0ad0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589d89f640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e0a0640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c58f0 0x56337e3d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f589e8a1640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e3cdbf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5890019e70 tx=0x7f589000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e4b6bd3ba12d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f5881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f5881ffb640 1 -- 172.21.15.99:0/2606943189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58900190d0 con 0x56337e3d0ad0 2024-05-14T17:09:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f5881ffb640 1 -- 172.21.15.99:0/2606943189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f5890019270 con 0x56337e3d0ad0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58940c9780 con 0x56337e3d0ad0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f5881ffb640 1 -- 172.21.15.99:0/2606943189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f589002b430 con 0x56337e3d0ad0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56337e3dee40 con 0x56337e3d0ad0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f5881ffb640 1 -- 172.21.15.99:0/2606943189 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58940c9780 con 0x56337e3d0ad0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f5881ffb640 1 -- 172.21.15.99:0/2606943189 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f58900177e0 con 0x56337e3d0ad0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 20 client.15912 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e3e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 4 client.15912 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.060+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56337e33ab80 con 0x56337e3e0410 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.061+0000 7f589e0a0640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e3e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.061+0000 7f5881ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.061+0000 7f589e0a0640 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e3e2890 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acfbde05e36dfe5d server_cookie=47d57784ffdeebba in_seq=0 out_seq=0 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f5881ffb640 1 -- 172.21.15.99:0/2606943189 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56337e33ab80 con 0x56337e3e0410 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f5881ffb640 10 client.15912 handle_command_reply: tid=0 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 client.15912 shutdown 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 msgr2=0x56337e3cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e3cdbf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5890019e70 tx=0x7f589000e040 comp rx=0 tx=0).stop 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 shutdown_connections 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e3e2890 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f5881ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e3cdbf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 --2- 172.21.15.99:0/2606943189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c58f0 0x56337e3d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 >> 172.21.15.99:0/2606943189 conn(0x56337e3c1e00 msgr2=0x56337e3c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 shutdown_connections 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 1 -- 172.21.15.99:0/2606943189 wait complete. 2024-05-14T17:09:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.062+0000 7f58a3016740 20 client.15912 trim_cache size 0 max 16384 2024-05-14T17:09:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 Processor -- start 2024-05-14T17:09:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 -- start start 2024-05-14T17:09:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 0x56337e3e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58981b0ed0 con 0x56337e3c58f0 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ae7a0 con 0x56337e3d0ad0 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f58a3016740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ac5f0 con 0x56337e3c6210 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f589e0a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 0x56337e3e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f589e0a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 0x56337e3e8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50896/0 (socket says 172.21.15.99:50896) 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.078+0000 7f589e0a0640 1 -- 172.21.15.99:0/383985643 learned_addr learned my addr 172.21.15.99:0/383985643 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589d89f640 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e0a0640 1 -- 172.21.15.99:0/383985643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 msgr2=0x56337e3e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e0a0640 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e8a1640 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e0a0640 1 -- 172.21.15.99:0/383985643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 msgr2=0x56337e3ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e0a0640 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e0a0640 1 -- 172.21.15.99:0/383985643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337e3c1c30 con 0x56337e3c58f0 2024-05-14T17:09:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589d89f640 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f589e0a0640 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 0x56337e3e8160 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5894100190 tx=0x7f5894102040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1569367d07fecc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f5883fff640 1 -- 172.21.15.99:0/383985643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940a5920 con 0x56337e3c58f0 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f5883fff640 1 -- 172.21.15.99:0/383985643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58940c9780 con 0x56337e3c58f0 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.079+0000 7f5883fff640 1 -- 172.21.15.99:0/383985643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940c93e0 con 0x56337e3c58f0 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 -- 172.21.15.99:0/383985643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 msgr2=0x56337e3e8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 0x56337e3e8160 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5894100190 tx=0x7f5894102040 comp rx=0 tx=0).stop 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 -- 172.21.15.99:0/383985643 shutdown_connections 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3d0ad0 0x56337e3ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c6210 0x56337e3e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 --2- 172.21.15.99:0/383985643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3c58f0 0x56337e3e8160 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 -- 172.21.15.99:0/383985643 >> 172.21.15.99:0/383985643 conn(0x56337e3c1e00 msgr2=0x56337e3db380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 -- 172.21.15.99:0/383985643 shutdown_connections 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 -- 172.21.15.99:0/383985643 wait complete. 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 -- start start 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.080+0000 7f58a3016740 1 Processor -- start 2024-05-14T17:09:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3d69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c6210 0x56337e3d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e449d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58981acc60 con 0x56337e3d0ad0 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ae0d0 con 0x56337e3c6210 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58981ad330 con 0x56337e3c58f0 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 10 client.0 _ll_register_callbacks cb 0x7f5890019b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e449d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e449d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50910/0 (socket says 172.21.15.99:50910) 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 -- 172.21.15.99:0/4216906426 learned_addr learned my addr 172.21.15.99:0/4216906426 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e0a0640 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3d69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 -- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 msgr2=0x56337e3d69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3d69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 -- 172.21.15.99:0/4216906426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c6210 msgr2=0x56337e3d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c6210 0x56337e3d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 -- 172.21.15.99:0/4216906426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337e3c92b0 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f589e8a1640 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e449d20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5890017e40 tx=0x7f5890019380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c441a338d18271a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f5881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f5881ffb640 1 -- 172.21.15.99:0/4216906426 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890017630 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f5881ffb640 1 -- 172.21.15.99:0/4216906426 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f58900177d0 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58940c9780 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f5881ffb640 1 -- 172.21.15.99:0/4216906426 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58900179d0 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 10 client.15918 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.081+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56337e44b2e0 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f5881ffb640 1 -- 172.21.15.99:0/4216906426 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58940c9780 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f5881ffb640 1 -- 172.21.15.99:0/4216906426 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5890032070 con 0x56337e3d0ad0 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f58a3016740 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f58a3016740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f58a3016740 20 client.15918 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f58a3016740 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e44dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f58a3016740 4 client.15918 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56337e334360 con 0x56337e3e0410 2024-05-14T17:09:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f589e0a0640 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e44dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f5881ffb640 10 client.15918 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.082+0000 7f589e0a0640 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e44dac0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecba7d61dc15294a server_cookie=9809334127e5bcee in_seq=0 out_seq=0 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f5881ffb640 1 -- 172.21.15.99:0/4216906426 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56337e334360 con 0x56337e3e0410 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f5881ffb640 10 client.15918 handle_command_reply: tid=0 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 client.15918 shutdown 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 msgr2=0x56337e449d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e449d20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5890017e40 tx=0x7f5890019380 comp rx=0 tx=0).stop 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 shutdown_connections 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56337e3e0410 0x56337e44dac0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56337e3d0ad0 0x56337e449d20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56337e3c6210 0x56337e3d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 --2- 172.21.15.99:0/4216906426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56337e3c58f0 0x56337e3d69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f5881ffb640 0 client.15918 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 >> 172.21.15.99:0/4216906426 conn(0x56337e3c1e00 msgr2=0x56337e3dbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 shutdown_connections 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 1 -- 172.21.15.99:0/4216906426 wait complete. 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.107+0000 7f58a3016740 20 client.15918 trim_cache size 0 max 16384 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:09:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58840b2a10 msgr2=0x7f58840b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58840b2a10 0x7f58840b4ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f589000a5a0 tx=0x7f589000c040 comp rx=0 tx=0).stop 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 msgr2=0x7f5898199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 0x7f5898199210 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5894004770 tx=0x7f58940042f0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 shutdown_connections 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f58840b2a10 0x7f58840b4ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5898108a70 0x7f589819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5898103420 0x7f5898199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 --2- 172.21.15.99:0/4058762759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5898102a60 0x7f5898199210 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 >> 172.21.15.99:0/4058762759 conn(0x7f5898079ee0 msgr2=0x7f58981003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 shutdown_connections 2024-05-14T17:09:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:05.108+0000 7f58a0304640 1 -- 172.21.15.99:0/4058762759 wait complete. 2024-05-14T17:09:05.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:04 smithi150 ceph-mon[36204]: pgmap v166: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.5 MiB/s wr, 131 op/s 2024-05-14T17:09:05.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.59 MB/sec execute 77 sec latency 22999.392 ms 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 -- 172.21.15.99:0/4114048385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 msgr2=0x7fbbb41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/4114048385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb41081c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbba80099c0 tx=0x7fbba8009cd0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 -- 172.21.15.99:0/4114048385 shutdown_connections 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/4114048385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb41088f0 0x7fbbb4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/4114048385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb41037f0 0x7fbbb4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/4114048385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb41081c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 -- 172.21.15.99:0/4114048385 >> 172.21.15.99:0/4114048385 conn(0x7fbbb407a6a0 msgr2=0x7fbbb407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 -- 172.21.15.99:0/4114048385 shutdown_connections 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 -- 172.21.15.99:0/4114048385 wait complete. 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.120+0000 7fbbba2f8700 1 Processor -- start 2024-05-14T17:09:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 -- start start 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb41037f0 0x7fbbb4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb41088f0 0x7fbbb40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41136d0 con 0x7fbbb41041a0 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4113550 con 0x7fbbb41037f0 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbba2f8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4113850 con 0x7fbbb41088f0 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb4075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50920/0 (socket says 172.21.15.99:50920) 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 -- 172.21.15.99:0/348370303 learned_addr learned my addr 172.21.15.99:0/348370303 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb3fff700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb41037f0 0x7fbbb4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb8895700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb41088f0 0x7fbbb40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 -- 172.21.15.99:0/348370303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb41088f0 msgr2=0x7fbbb40762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb41088f0 0x7fbbb40762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 -- 172.21.15.99:0/348370303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb41037f0 msgr2=0x7fbbb4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb41037f0 0x7fbbb4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.121+0000 7fbbb37fe700 1 -- 172.21.15.99:0/348370303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb41a7ff0 con 0x7fbbb41041a0 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.122+0000 7fbbb37fe700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb4075d80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbba80098e0 tx=0x7fbba80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.122+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8027070 con 0x7fbbb41041a0 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.122+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbba8003f20 con 0x7fbbb41041a0 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.122+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb41a8280 con 0x7fbbb41041a0 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.122+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8022d70 con 0x7fbbb41041a0 2024-05-14T17:09:06.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.122+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb41a8730 con 0x7fbbb41041a0 2024-05-14T17:09:06.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.123+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb406a230 con 0x7fbbb41041a0 2024-05-14T17:09:06.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.124+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbba802a070 con 0x7fbbb41041a0 2024-05-14T17:09:06.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.125+0000 7fbbb17fa700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb9c0b2a70 0x7fbb9c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.125+0000 7fbbb3fff700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb9c0b2a70 0x7fbb9c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.125+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fbba80e3c00 con 0x7fbbb41041a0 2024-05-14T17:09:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.125+0000 7fbbb3fff700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb9c0b2a70 0x7fbb9c0b4f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbba400ac50 tx=0x7fbba400a380 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.129+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbba8031020 con 0x7fbbb41041a0 2024-05-14T17:09:06.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.379+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbb402e560 con 0x7fbbb41041a0 2024-05-14T17:09:06.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.380+0000 7fbbb17fa700 1 -- 172.21.15.99:0/348370303 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11876 (secure 0 0 0) 0x7fbba8013070 con 0x7fbbb41041a0 2024-05-14T17:09:06.380 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:09:06.380 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:06:34.984717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.382+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb9c0b2a70 msgr2=0x7fbb9c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.382+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb9c0b2a70 0x7fbb9c0b4f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbba400ac50 tx=0x7fbba400a380 comp rx=0 tx=0).stop 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.382+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 msgr2=0x7fbbb4075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.382+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb4075d80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbba80098e0 tx=0x7fbba80057c0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 shutdown_connections 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbb9c0b2a70 0x7fbb9c0b4f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbb41088f0 0x7fbbb40762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb41037f0 0x7fbbb4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 --2- 172.21.15.99:0/348370303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb41041a0 0x7fbbb4075d80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 >> 172.21.15.99:0/348370303 conn(0x7fbbb407a6a0 msgr2=0x7fbbb41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 shutdown_connections 2024-05-14T17:09:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:06.383+0000 7fbbba2f8700 1 -- 172.21.15.99:0/348370303 wait complete. 2024-05-14T17:09:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-05-14T17:09:06.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.48 MB/sec execute 78 sec latency 23999.442 ms 2024-05-14T17:09:07.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:06 smithi099 ceph-mon[31851]: pgmap v167: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-05-14T17:09:07.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:06 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/348370303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:09:07.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:06 smithi193 ceph-mon[35962]: pgmap v167: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-05-14T17:09:07.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:06 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/348370303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:09:07.127 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 3942b6fd --release --await' 2024-05-14T17:09:07.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 3942b6fd --release --await 2024-05-14T17:09:07.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:06 smithi150 ceph-mon[36204]: pgmap v167: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-05-14T17:09:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:06 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/348370303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:09:07.371 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:09:07.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.38 MB/sec execute 79 sec latency 24999.481 ms 2024-05-14T17:09:08.527 INFO:tasks.workunit.client.0.smithi099.stdout: 1 261865 8.27 MB/sec execute 80 sec latency 25999.547 ms 2024-05-14T17:09:08.832 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:08 smithi099 ceph-mon[31851]: pgmap v168: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-05-14T17:09:09.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:08 smithi193 ceph-mon[35962]: pgmap v168: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-05-14T17:09:09.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 -- 172.21.15.99:0/2653816632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe081077d0 msgr2=0x7efe0810f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 --2- 172.21.15.99:0/2653816632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe081077d0 0x7efe0810f370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7efe000099c0 tx=0x7efe00009cd0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 -- 172.21.15.99:0/2653816632 shutdown_connections 2024-05-14T17:09:09.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 --2- 172.21.15.99:0/2653816632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe081077d0 0x7efe0810f370 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 --2- 172.21.15.99:0/2653816632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 0x7efe08102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 --2- 172.21.15.99:0/2653816632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe08103430 0x7efe081070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 -- 172.21.15.99:0/2653816632 >> 172.21.15.99:0/2653816632 conn(0x7efe080793f0 msgr2=0x7efe08079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:09.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 -- 172.21.15.99:0/2653816632 shutdown_connections 2024-05-14T17:09:09.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.067+0000 7efe0da74700 1 -- 172.21.15.99:0/2653816632 wait complete. 2024-05-14T17:09:09.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 Processor -- start 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 -- start start 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 0x7efe0819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 0x7efe0819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe081077d0 0x7efe08199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe08112660 con 0x7efe08103430 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe081124e0 con 0x7efe081077d0 2024-05-14T17:09:09.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe0da74700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe081127e0 con 0x7efe08102a80 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 0x7efe0819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.068+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 0x7efe0819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50934/0 (socket says 172.21.15.99:50934) 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 -- 172.21.15.99:0/4154990225 learned_addr learned my addr 172.21.15.99:0/4154990225 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe07fff700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe081077d0 0x7efe08199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe077fe700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 0x7efe0819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 -- 172.21.15.99:0/4154990225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 msgr2=0x7efe0819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 0x7efe0819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 -- 172.21.15.99:0/4154990225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe081077d0 msgr2=0x7efe08199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe081077d0 0x7efe08199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 -- 172.21.15.99:0/4154990225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0819a160 con 0x7efe08103430 2024-05-14T17:09:09.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe077fe700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 0x7efe0819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:09.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe06ffd700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 0x7efe0819fd80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00d950 tx=0x7efdfc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe04ff9700 1 -- 172.21.15.99:0/4154990225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc017070 con 0x7efe08103430 2024-05-14T17:09:09.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe04ff9700 1 -- 172.21.15.99:0/4154990225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc009de0 con 0x7efe08103430 2024-05-14T17:09:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe04ff9700 1 -- 172.21.15.99:0/4154990225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00f3c0 con 0x7efe08103430 2024-05-14T17:09:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0819a450 con 0x7efe08103430 2024-05-14T17:09:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.069+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe081a6710 con 0x7efe08103430 2024-05-14T17:09:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.072+0000 7efe04ff9700 1 -- 172.21.15.99:0/4154990225 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efdfc01a070 con 0x7efe08103430 2024-05-14T17:09:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.072+0000 7efe04ff9700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efdf00b29f0 0x7efdf00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.072+0000 7efe077fe700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efdf00b29f0 0x7efdf00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe04ff9700 1 -- 172.21.15.99:0/4154990225 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efdfc0e1df0 con 0x7efe08103430 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe077fe700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efdf00b29f0 0x7efdf00b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdf8007fd0 tx=0x7efdf8007810 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 Processor -- start 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 -- start start 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c7da0 0x563d2b2c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c86c0 0x563d2b2c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563d2b220d90 con 0x563d2b2c2ad0 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563d2b210930 con 0x563d2b2c86c0 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe1830ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563d2b214970 con 0x563d2b2c7da0 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe07fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe07fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50942/0 (socket says 172.21.15.99:50942) 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.073+0000 7efe07fff700 1 -- 172.21.15.99:0/2316028862 learned_addr learned my addr 172.21.15.99:0/2316028862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe06ffd700 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c86c0 0x563d2b2c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe077fe700 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c7da0 0x563d2b2c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe07fff700 1 -- 172.21.15.99:0/2316028862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c7da0 msgr2=0x563d2b2c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe07fff700 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c7da0 0x563d2b2c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe07fff700 1 -- 172.21.15.99:0/2316028862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c86c0 msgr2=0x563d2b2c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe07fff700 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c86c0 0x563d2b2c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe07fff700 1 -- 172.21.15.99:0/2316028862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2b1609d0 con 0x563d2b2c2ad0 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efe07fff700 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c2f50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efe00015000 tx=0x7efe00005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999b9560846bf880 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efdedffb700 1 -- 172.21.15.99:0/2316028862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00003a60 con 0x563d2b2c2ad0 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efdedffb700 1 -- 172.21.15.99:0/2316028862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe00026070 con 0x563d2b2c2ad0 2024-05-14T17:09:09.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.074+0000 7efdedffb700 1 -- 172.21.15.99:0/2316028862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000b9f0 con 0x563d2b2c2ad0 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 -- 172.21.15.99:0/2316028862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 msgr2=0x563d2b2c2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c2f50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efe00015000 tx=0x7efe00005c90 comp rx=0 tx=0).stop 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 -- 172.21.15.99:0/2316028862 shutdown_connections 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c2f50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c7da0 0x563d2b2c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2316028862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c86c0 0x563d2b2c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 -- 172.21.15.99:0/2316028862 >> 172.21.15.99:0/2316028862 conn(0x563d2b1ed350 msgr2=0x563d2b2bcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 -- 172.21.15.99:0/2316028862 shutdown_connections 2024-05-14T17:09:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 -- 172.21.15.99:0/2316028862 wait complete. 2024-05-14T17:09:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 -- start start 2024-05-14T17:09:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.075+0000 7efe1830ab80 1 Processor -- start 2024-05-14T17:09:09.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c2ad0 0x563d2b2ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 0x563d2b2cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563d2b1170a0 con 0x563d2b2c86c0 2024-05-14T17:09:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563d2b1b28c0 con 0x563d2b2c7da0 2024-05-14T17:09:09.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563d2b2cf0a0 con 0x563d2b2c2ad0 2024-05-14T17:09:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe1830ab80 10 client.0 _ll_register_callbacks cb 0x563d2b1a7680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 0x563d2b2cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 0x563d2b2cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50944/0 (socket says 172.21.15.99:50944) 2024-05-14T17:09:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 -- 172.21.15.99:0/419336765 learned_addr learned my addr 172.21.15.99:0/419336765 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe077fe700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c2ad0 0x563d2b2ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe06ffd700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 -- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c2ad0 msgr2=0x563d2b2ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c2ad0 0x563d2b2ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 -- 172.21.15.99:0/419336765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 msgr2=0x563d2b2cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 -- 172.21.15.99:0/419336765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2b1af790 con 0x563d2b2c86c0 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe06ffd700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:09.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efe07fff700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 0x563d2b2cbbf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efe00009d10 tx=0x7efe00005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0be209fa1aa7336 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.076+0000 7efde6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efde6ffd700 1 -- 172.21.15.99:0/419336765 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000b990 con 0x563d2b2c86c0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efde6ffd700 1 -- 172.21.15.99:0/419336765 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efe0000bb30 con 0x563d2b2c86c0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe00026070 con 0x563d2b2c86c0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efde6ffd700 1 -- 172.21.15.99:0/419336765 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe0000bd30 con 0x563d2b2c86c0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 10 client.15936 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d2b2d9700 con 0x563d2b2c86c0 2024-05-14T17:09:09.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efde6ffd700 1 -- 172.21.15.99:0/419336765 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe00026070 con 0x563d2b2c86c0 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efde6ffd700 1 -- 172.21.15.99:0/419336765 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efe0000fb60 con 0x563d2b2c86c0 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 10 client.15936 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 20 client.15936 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b2dbbd0 0x563d2b2ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 4 client.15936 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.077+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d2b28e460 con 0x563d2b2dbbd0 2024-05-14T17:09:09.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.078+0000 7efe077fe700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b2dbbd0 0x563d2b2ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.078+0000 7efde6ffd700 10 client.15936 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.078+0000 7efe077fe700 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b2dbbd0 0x563d2b2ddfc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ce85b9e8243f84 server_cookie=3d874cc30564bbc7 in_seq=0 out_seq=0 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efde6ffd700 1 -- 172.21.15.99:0/419336765 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d2b28e460 con 0x563d2b2dbbd0 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efde6ffd700 10 client.15936 handle_command_reply: tid=0 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 client.15936 shutdown 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 msgr2=0x563d2b2cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 0x563d2b2cbbf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efe00009d10 tx=0x7efe00005180 comp rx=0 tx=0).stop 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 shutdown_connections 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b2dbbd0 0x563d2b2ddfc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c86c0 0x563d2b2cbbf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c2ad0 0x563d2b2ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 --2- 172.21.15.99:0/419336765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 >> 172.21.15.99:0/419336765 conn(0x563d2b1ed350 msgr2=0x563d2b2b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efde6ffd700 0 client.15936 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 shutdown_connections 2024-05-14T17:09:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 1 -- 172.21.15.99:0/419336765 wait complete. 2024-05-14T17:09:09.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.079+0000 7efe1830ab80 20 client.15936 trim_cache size 0 max 16384 2024-05-14T17:09:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 Processor -- start 2024-05-14T17:09:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 -- start start 2024-05-14T17:09:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b2b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe081b0550 con 0x563d2b2c2ad0 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe081adde0 con 0x563d2b2c7da0 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.099+0000 7efe1830ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe081abc30 con 0x563d2b2c86c0 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe07fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b2b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50956/0 (socket says 172.21.15.99:50956) 2024-05-14T17:09:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 -- 172.21.15.99:0/3442800143 learned_addr learned my addr 172.21.15.99:0/3442800143 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 -- 172.21.15.99:0/3442800143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 msgr2=0x563d2b2b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b2b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 -- 172.21.15.99:0/3442800143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 msgr2=0x563d2b2ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 -- 172.21.15.99:0/3442800143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2b2cdf50 con 0x563d2b2c2ad0 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efe077fe700 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c24b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efe0819ab40 tx=0x7efdf8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87412c5ef28dc6be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efded7fa700 1 -- 172.21.15.99:0/3442800143 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8004db0 con 0x563d2b2c2ad0 2024-05-14T17:09:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efded7fa700 1 -- 172.21.15.99:0/3442800143 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf8028610 con 0x563d2b2c2ad0 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.100+0000 7efded7fa700 1 -- 172.21.15.99:0/3442800143 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf80054c0 con 0x563d2b2c2ad0 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 -- 172.21.15.99:0/3442800143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 msgr2=0x563d2b2c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c24b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efe0819ab40 tx=0x7efdf8011000 comp rx=0 tx=0).stop 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 -- 172.21.15.99:0/3442800143 shutdown_connections 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b2b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c2ad0 0x563d2b2c24b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 --2- 172.21.15.99:0/3442800143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c7da0 0x563d2b2ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 -- 172.21.15.99:0/3442800143 >> 172.21.15.99:0/3442800143 conn(0x563d2b1ed350 msgr2=0x563d2b2c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 -- 172.21.15.99:0/3442800143 shutdown_connections 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 -- 172.21.15.99:0/3442800143 wait complete. 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.101+0000 7efe1830ab80 1 -- start start 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 Processor -- start 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c2ad0 0x563d2b2dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 0x563d2b2df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b3450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe081ac2a0 con 0x563d2b2c7da0 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe081ad710 con 0x563d2b2c2ad0 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe081ac970 con 0x563d2b2c86c0 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe1830ab80 10 client.0 _ll_register_callbacks cb 0x563d2b1a7680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 0x563d2b2df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 0x563d2b2df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50972/0 (socket says 172.21.15.99:50972) 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 -- 172.21.15.99:0/2180521004 learned_addr learned my addr 172.21.15.99:0/2180521004 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:09.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe077fe700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c2ad0 0x563d2b2dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe07fff700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b3450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 -- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 msgr2=0x563d2b3450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b3450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 -- 172.21.15.99:0/2180521004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c2ad0 msgr2=0x563d2b2dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c2ad0 0x563d2b2dee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe06ffd700 1 -- 172.21.15.99:0/2180521004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2b345630 con 0x563d2b2c7da0 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.102+0000 7efe07fff700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b3450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe06ffd700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 0x563d2b2df3b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0a6370 tx=0x7efdfc0a9640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd880ad7a6e31e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efde6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efde6ffd700 1 -- 172.21.15.99:0/2180521004 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc021040 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efde6ffd700 1 -- 172.21.15.99:0/2180521004 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7efdfc009930 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf8028610 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efde6ffd700 1 -- 172.21.15.99:0/2180521004 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdfc0a9800 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 10 client.15942 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d2b346910 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efde6ffd700 1 -- 172.21.15.99:0/2180521004 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf8028610 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efde6ffd700 1 -- 172.21.15.99:0/2180521004 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efdfc0a4b90 con 0x563d2b2c7da0 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:09.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 10 client.15942 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 20 client.15942 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b347050 0x563d2b349470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 4 client.15942 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3942b6fd", "release": true, "await": true}] 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.103+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3942b6fd", "release": true, "await": true}) -- 0x563d2b222ce0 con 0x563d2b347050 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.104+0000 7efe077fe700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b347050 0x563d2b349470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.104+0000 7efde6ffd700 10 client.15942 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:09.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.104+0000 7efe077fe700 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b347050 0x563d2b349470 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ad44955c316832 server_cookie=1ddd863c7dba2346 in_seq=0 out_seq=0 2024-05-14T17:09:09.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.112+0000 7efde6ffd700 1 -- 172.21.15.99:0/2180521004 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d2b222ce0 con 0x563d2b347050 2024-05-14T17:09:09.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.112+0000 7efde6ffd700 10 client.15942 handle_command_reply: tid=0 2024-05-14T17:09:09.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.112+0000 7efe1830ab80 1 client.15942 shutdown 2024-05-14T17:09:09.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 msgr2=0x563d2b2df3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 0x563d2b2df3b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0a6370 tx=0x7efdfc0a9640 comp rx=0 tx=0).stop 2024-05-14T17:09:09.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 shutdown_connections 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d2b2c86c0 0x563d2b3450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563d2b2c2ad0 0x563d2b2dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563d2b347050 0x563d2b349470 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efde6ffd700 0 client.15942 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 --2- 172.21.15.99:0/2180521004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d2b2c7da0 0x563d2b2df3b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 >> 172.21.15.99:0/2180521004 conn(0x563d2b1ed350 msgr2=0x563d2b2ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 shutdown_connections 2024-05-14T17:09:09.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 1 -- 172.21.15.99:0/2180521004 wait complete. 2024-05-14T17:09:09.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe1830ab80 20 client.15942 trim_cache size 0 max 16384 2024-05-14T17:09:09.120 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:09:09.120 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 27, 2024-05-14T17:09:09.120 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 16, 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "3942b6fd": { 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 16, 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:09:09.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 146.8, 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:09:09.122 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:09:09.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:09:09.124 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:09:09.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efdf00b29f0 msgr2=0x7efdf00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efdf00b29f0 0x7efdf00b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdf8007fd0 tx=0x7efdf8007810 comp rx=0 tx=0).stop 2024-05-14T17:09:09.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 msgr2=0x7efe0819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:09.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 0x7efe0819fd80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00d950 tx=0x7efdfc00dc60 comp rx=0 tx=0).stop 2024-05-14T17:09:09.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 shutdown_connections 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7efdf00b29f0 0x7efdf00b4eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe081077d0 0x7efe08199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe08102a80 0x7efe0819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 --2- 172.21.15.99:0/4154990225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08103430 0x7efe0819fd80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.113+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 >> 172.21.15.99:0/4154990225 conn(0x7efe080793f0 msgr2=0x7efe0806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.114+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 shutdown_connections 2024-05-14T17:09:09.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:09:09.114+0000 7efe0da74700 1 -- 172.21.15.99:0/4154990225 wait complete. 2024-05-14T17:09:09.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:08 smithi150 ceph-mon[36204]: pgmap v168: 97 pgs: 97 active+clean; 67 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-05-14T17:09:09.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 262731 8.25 MB/sec execute 81 sec latency 26587.396 ms 2024-05-14T17:09:09.902 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3942b6fd', total seconds elapsed: 32.6 2024-05-14T17:09:10.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 264914 8.24 MB/sec execute 82 sec latency 23.149 ms 2024-05-14T17:09:11.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:10 smithi099 ceph-mon[31851]: pgmap v169: 97 pgs: 97 active+clean; 71 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.5 MiB/s wr, 116 op/s 2024-05-14T17:09:11.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:10 smithi193 ceph-mon[35962]: pgmap v169: 97 pgs: 97 active+clean; 71 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.5 MiB/s wr, 116 op/s 2024-05-14T17:09:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:10 smithi150 ceph-mon[36204]: pgmap v169: 97 pgs: 97 active+clean; 71 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 2.5 MiB/s wr, 116 op/s 2024-05-14T17:09:11.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 266782 8.29 MB/sec execute 83 sec latency 24.153 ms 2024-05-14T17:09:12.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 269548 8.40 MB/sec execute 84 sec latency 32.490 ms 2024-05-14T17:09:13.004 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:12 smithi150 ceph-mon[36204]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-05-14T17:09:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:12 smithi099 ceph-mon[31851]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-05-14T17:09:13.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:12 smithi193 ceph-mon[35962]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.6 MiB/s wr, 87 op/s 2024-05-14T17:09:13.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 271378 8.38 MB/sec execute 85 sec latency 14.299 ms 2024-05-14T17:09:14.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 273731 8.44 MB/sec execute 86 sec latency 34.888 ms 2024-05-14T17:09:15.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:14 smithi099 ceph-mon[31851]: pgmap v171: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 222 op/s 2024-05-14T17:09:15.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:14 smithi193 ceph-mon[35962]: pgmap v171: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 222 op/s 2024-05-14T17:09:15.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:14 smithi150 ceph-mon[36204]: pgmap v171: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 222 op/s 2024-05-14T17:09:15.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 276361 8.50 MB/sec execute 87 sec latency 18.920 ms 2024-05-14T17:09:16.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 8.53 MB/sec execute 88 sec latency 84.649 ms 2024-05-14T17:09:17.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:16 smithi099 ceph-mon[31851]: pgmap v172: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-14T17:09:17.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:16 smithi193 ceph-mon[35962]: pgmap v172: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-14T17:09:17.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:16 smithi150 ceph-mon[36204]: pgmap v172: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-14T17:09:17.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 8.43 MB/sec execute 89 sec latency 1084.703 ms 2024-05-14T17:09:18.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 -- 172.21.15.99:0/3872762160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 msgr2=0x7fc8601013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 --2- 172.21.15.99:0/3872762160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc8601013d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc848009930 tx=0x7fc84802a420 comp rx=0 tx=0).stop 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 -- 172.21.15.99:0/3872762160 shutdown_connections 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 --2- 172.21.15.99:0/3872762160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860101910 0x7fc86010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 --2- 172.21.15.99:0/3872762160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc8601013d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 --2- 172.21.15.99:0/3872762160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860107a30 0x7fc860107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 -- 172.21.15.99:0/3872762160 >> 172.21.15.99:0/3872762160 conn(0x7fc860077ed0 msgr2=0x7fc8600ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 -- 172.21.15.99:0/3872762160 shutdown_connections 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 -- 172.21.15.99:0/3872762160 wait complete. 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 Processor -- start 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.207+0000 7fc8665da640 1 -- start start 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc860102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860101910 0x7fc860102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860107a30 0x7fc860102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc860112db0 con 0x7fc860100f50 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc860112c30 con 0x7fc860101910 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc860112f30 con 0x7fc860107a30 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc860102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc860102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49630/0 (socket says 172.21.15.99:49630) 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 -- 172.21.15.99:0/3439941678 learned_addr learned my addr 172.21.15.99:0/3439941678 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc857fff640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860101910 0x7fc860102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc864b77640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860107a30 0x7fc860102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 -- 172.21.15.99:0/3439941678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860107a30 msgr2=0x7fc860102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860107a30 0x7fc860102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 -- 172.21.15.99:0/3439941678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860101910 msgr2=0x7fc860102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860101910 0x7fc860102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 -- 172.21.15.99:0/3439941678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8601a7770 con 0x7fc860100f50 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc864b77640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860107a30 0x7fc860102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc857fff640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860101910 0x7fc860102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85ffff640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc860102130 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d950 tx=0x7fc85000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85dffb640 1 -- 172.21.15.99:0/3439941678 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850017070 con 0x7fc860100f50 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85dffb640 1 -- 172.21.15.99:0/3439941678 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc850009c30 con 0x7fc860100f50 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc85dffb640 1 -- 172.21.15.99:0/3439941678 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000f620 con 0x7fc860100f50 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8601a7a60 con 0x7fc860100f50 2024-05-14T17:09:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.208+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8601a7f80 con 0x7fc860100f50 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc85dffb640 1 -- 172.21.15.99:0/3439941678 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc850010460 con 0x7fc860100f50 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 Processor -- start 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 -- start start 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cd7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4ce4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f4f4c4ac30 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4f4c3a7d0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc8693d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f4f4c3e810 con 0x55f4f4cd8210 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc857fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4ce4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc857fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4ce4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49640/0 (socket says 172.21.15.99:49640) 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc857fff640 1 -- 172.21.15.99:0/2644038988 learned_addr learned my addr 172.21.15.99:0/2644038988 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc864b77640 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cd7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.211+0000 7fc85ffff640 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc857fff640 1 -- 172.21.15.99:0/2644038988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd8210 msgr2=0x55f4f4ce2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc857fff640 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc857fff640 1 -- 172.21.15.99:0/2644038988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd78f0 msgr2=0x55f4f4cd7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc857fff640 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cd7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc857fff640 1 -- 172.21.15.99:0/2644038988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f494f210 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc85ffff640 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc85dffb640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc84c0b2a10 0x7fc84c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc857fff640 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4ce4ec0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc84802a940 tx=0x7fc848032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2662aad39779e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc864b77640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc84c0b2a10 0x7fc84c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc85dffb640 1 -- 172.21.15.99:0/3439941678 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fc8500e2240 con 0x7fc860100f50 2024-05-14T17:09:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc8567fc640 1 -- 172.21.15.99:0/2644038988 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84803f070 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc8567fc640 1 -- 172.21.15.99:0/2644038988 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8480316a0 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.212+0000 7fc8567fc640 1 -- 172.21.15.99:0/2644038988 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848035070 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 -- 172.21.15.99:0/2644038988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 msgr2=0x55f4f4ce4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4ce4ec0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc84802a940 tx=0x7fc848032cd0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc864b77640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc84c0b2a10 0x7fc84c0b4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc858005fd0 tx=0x7fc858009790 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 -- 172.21.15.99:0/2644038988 shutdown_connections 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4ce4ec0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2644038988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cd7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 -- 172.21.15.99:0/2644038988 >> 172.21.15.99:0/2644038988 conn(0x55f4f4cd3e00 msgr2=0x55f4f4cd41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 -- 172.21.15.99:0/2644038988 shutdown_connections 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 -- 172.21.15.99:0/2644038988 wait complete. 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.213+0000 7fc8693d0740 1 -- start start 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 Processor -- start 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4cdf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f4f4920790 con 0x55f4f4cd78f0 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4f4bf6890 con 0x55f4f4cd8210 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f4f4ce5740 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc8693d0740 10 client.0 _ll_register_callbacks cb 0x55f4f4bf9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49652/0 (socket says 172.21.15.99:49652) 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 -- 172.21.15.99:0/4011319863 learned_addr learned my addr 172.21.15.99:0/4011319863 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc864b77640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc857fff640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4cdf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 -- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4ce2ad0 msgr2=0x55f4f4cdfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 -- 172.21.15.99:0/4011319863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 msgr2=0x55f4f4cdf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4cdf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 -- 172.21.15.99:0/4011319863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f4a1b620 con 0x55f4f4cd78f0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc85ffff640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce2460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc8500a9420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ea2bd0421e18dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.214+0000 7fc83f7fe640 1 -- 172.21.15.99:0/4011319863 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850020050 con 0x55f4f4cd78f0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc83f7fe640 1 -- 172.21.15.99:0/4011319863 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fc8500e5c60 con 0x55f4f4cd78f0 2024-05-14T17:09:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8480316a0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc83f7fe640 1 -- 172.21.15.99:0/4011319863 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8500e5e60 con 0x55f4f4cd78f0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 10 client.15954 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4f4cf11d0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc83f7fe640 1 -- 172.21.15.99:0/4011319863 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8480316a0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc83f7fe640 1 -- 172.21.15.99:0/4011319863 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc8500a4ec0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 20 client.15954 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4cf2690 0x55f4f4cf4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 4 client.15954 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f4c4cb80 con 0x55f4f4cf2690 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.215+0000 7fc857fff640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4cf2690 0x55f4f4cf4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.216+0000 7fc83f7fe640 10 client.15954 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.216+0000 7fc857fff640 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4cf2690 0x55f4f4cf4ae0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b26621c3d76363 server_cookie=5323c56a565dec2a in_seq=0 out_seq=0 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.216+0000 7fc83f7fe640 1 -- 172.21.15.99:0/4011319863 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4f4c4cb80 con 0x55f4f4cf2690 2024-05-14T17:09:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.216+0000 7fc83f7fe640 10 client.15954 handle_command_reply: tid=0 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 client.15954 shutdown 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 msgr2=0x55f4f4ce2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce2460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc8500a9420 comp rx=0 tx=0).stop 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 shutdown_connections 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4cf2690 0x55f4f4cf4ae0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4cdf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc83f7fe640 0 client.15954 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 --2- 172.21.15.99:0/4011319863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce2460 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 >> 172.21.15.99:0/4011319863 conn(0x55f4f4cd3e00 msgr2=0x55f4f4cd3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 shutdown_connections 2024-05-14T17:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 1 -- 172.21.15.99:0/4011319863 wait complete. 2024-05-14T17:09:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.217+0000 7fc8693d0740 20 client.15954 trim_cache size 0 max 16384 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 Processor -- start 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 -- start start 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cfa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 0x55f4f4cfa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b2f80 con 0x55f4f4cd8210 2024-05-14T17:09:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b0850 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc8693d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601ae6a0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 0x55f4f4cfa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 0x55f4f4cfa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49666/0 (socket says 172.21.15.99:49666) 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 -- 172.21.15.99:0/2668539594 learned_addr learned my addr 172.21.15.99:0/2668539594 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc85ffff640 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cfa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 -- 172.21.15.99:0/2668539594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 msgr2=0x55f4f4cfa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cfa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 -- 172.21.15.99:0/2668539594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4ce2ad0 msgr2=0x55f4f4cdc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 -- 172.21.15.99:0/2668539594 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f4cd2f50 con 0x55f4f4cd8210 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.232+0000 7fc857fff640 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 0x55f4f4cfa930 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc848033000 tx=0x7fc848000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f4bda26dc94c8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc855ffb640 1 -- 172.21.15.99:0/2668539594 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848051070 con 0x55f4f4cd8210 2024-05-14T17:09:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc855ffb640 1 -- 172.21.15.99:0/2668539594 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc848004030 con 0x55f4f4cd8210 2024-05-14T17:09:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc855ffb640 1 -- 172.21.15.99:0/2668539594 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848035040 con 0x55f4f4cd8210 2024-05-14T17:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 -- 172.21.15.99:0/2668539594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 msgr2=0x55f4f4cfa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 0x55f4f4cfa930 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc848033000 tx=0x7fc848000bc0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 -- 172.21.15.99:0/2668539594 shutdown_connections 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4cdc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4cd8210 0x55f4f4cfa930 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 --2- 172.21.15.99:0/2668539594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4cfa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 -- 172.21.15.99:0/2668539594 >> 172.21.15.99:0/2668539594 conn(0x55f4f4cd3e00 msgr2=0x55f4f4ced600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 -- 172.21.15.99:0/2668539594 shutdown_connections 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.233+0000 7fc8693d0740 1 -- 172.21.15.99:0/2668539594 wait complete. 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 -- start start 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 Processor -- start 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4d5bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601aed10 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601b0180 con 0x55f4f4cd8210 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8601af3e0 con 0x55f4f4cd78f0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc8693d0740 10 client.0 _ll_register_callbacks cb 0x7fc85000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc864b77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4d5bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc864b77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4d5bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49676/0 (socket says 172.21.15.99:49676) 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc864b77640 1 -- 172.21.15.99:0/1390086118 learned_addr learned my addr 172.21.15.99:0/1390086118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc85ffff640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.234+0000 7fc857fff640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc864b77640 1 -- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 msgr2=0x55f4f4ce8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc864b77640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc864b77640 1 -- 172.21.15.99:0/1390086118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 msgr2=0x55f4f4ce9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc864b77640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc864b77640 1 -- 172.21.15.99:0/1390086118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f4ce5230 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc85ffff640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc864b77640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4d5bfb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc8580129f0 tx=0x7fc858004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254705ca1e7eb3ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc83f7fe640 1 -- 172.21.15.99:0/1390086118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858003d80 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc83f7fe640 1 -- 172.21.15.99:0/1390086118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fc85801f8c0 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc848004030 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc83f7fe640 1 -- 172.21.15.99:0/1390086118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc85801fac0 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 10 client.15960 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4f4d5d720 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc83f7fe640 1 -- 172.21.15.99:0/1390086118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc848004030 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc83f7fe640 1 -- 172.21.15.99:0/1390086118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc85800f040 con 0x55f4f4ce2ad0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 20 client.15960 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4d5df60 0x55f4f4d60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 4 client.15960 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.235+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4f4c46360 con 0x55f4f4d5df60 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.236+0000 7fc85ffff640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4d5df60 0x55f4f4d60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.236+0000 7fc83f7fe640 10 client.15960 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.236+0000 7fc85ffff640 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4d5df60 0x55f4f4d60380 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed90151166a9b8c server_cookie=925a9c59ac356009 in_seq=0 out_seq=0 2024-05-14T17:09:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.294+0000 7fc83f7fe640 1 -- 172.21.15.99:0/1390086118 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f4f4c46360 con 0x55f4f4d5df60 2024-05-14T17:09:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.294+0000 7fc83f7fe640 10 client.15960 handle_command_reply: tid=0 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.294+0000 7fc8693d0740 1 client.15960 shutdown 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 msgr2=0x55f4f4d5bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4d5bfb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc8580129f0 tx=0x7fc858004ab0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 shutdown_connections 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f4f4d5df60 0x55f4f4d60380 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4f4ce2ad0 0x55f4f4d5bfb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4f4cd8210 0x55f4f4ce9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc83f7fe640 0 client.15960 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 --2- 172.21.15.99:0/1390086118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f4cd78f0 0x55f4f4ce8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 >> 172.21.15.99:0/1390086118 conn(0x55f4f4cd3e00 msgr2=0x55f4f4cede90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 shutdown_connections 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 1 -- 172.21.15.99:0/1390086118 wait complete. 2024-05-14T17:09:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.295+0000 7fc8693d0740 20 client.15960 trim_cache size 0 max 16384 2024-05-14T17:09:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:09:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:09:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:09:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc84c0b2a10 msgr2=0x7fc84c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc84c0b2a10 0x7fc84c0b4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc858005fd0 tx=0x7fc858009790 comp rx=0 tx=0).stop 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 msgr2=0x7fc860102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc860102130 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d950 tx=0x7fc85000de10 comp rx=0 tx=0).stop 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 shutdown_connections 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc84c0b2a10 0x7fc84c0b4ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc860107a30 0x7fc860102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc860101910 0x7fc860102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 --2- 172.21.15.99:0/3439941678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc860100f50 0x7fc860102130 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 >> 172.21.15.99:0/3439941678 conn(0x7fc860077ed0 msgr2=0x7fc860105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 shutdown_connections 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:18.296+0000 7fc8665da640 1 -- 172.21.15.99:0/3439941678 wait complete. 2024-05-14T17:09:18.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 8.34 MB/sec execute 90 sec latency 2084.785 ms 2024-05-14T17:09:19.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:18 smithi099 ceph-mon[31851]: pgmap v173: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-14T17:09:19.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:18 smithi193 ceph-mon[35962]: pgmap v173: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-14T17:09:19.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:18 smithi150 ceph-mon[36204]: pgmap v173: 97 pgs: 97 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-14T17:09:19.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 8.25 MB/sec execute 91 sec latency 3084.880 ms 2024-05-14T17:09:20.528 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 8.16 MB/sec execute 92 sec latency 4084.976 ms 2024-05-14T17:09:21.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:20 smithi099 ceph-mon[31851]: pgmap v174: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-05-14T17:09:21.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:20 smithi193 ceph-mon[35962]: pgmap v174: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-05-14T17:09:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:20 smithi150 ceph-mon[36204]: pgmap v174: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-05-14T17:09:21.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 8.07 MB/sec execute 93 sec latency 5085.066 ms 2024-05-14T17:09:22.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 7.98 MB/sec execute 94 sec latency 6085.164 ms 2024-05-14T17:09:23.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:22 smithi099 ceph-mon[31851]: pgmap v175: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-05-14T17:09:23.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:22 smithi193 ceph-mon[35962]: pgmap v175: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-05-14T17:09:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:22 smithi150 ceph-mon[36204]: pgmap v175: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-05-14T17:09:23.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 7.90 MB/sec execute 95 sec latency 7085.253 ms 2024-05-14T17:09:24.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 278162 7.82 MB/sec execute 96 sec latency 8085.356 ms 2024-05-14T17:09:25.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:24 smithi193 ceph-mon[35962]: pgmap v176: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-05-14T17:09:25.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:24 smithi150 ceph-mon[36204]: pgmap v176: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-05-14T17:09:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:24 smithi099 ceph-mon[31851]: pgmap v176: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 239 op/s 2024-05-14T17:09:25.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 279252 7.77 MB/sec execute 97 sec latency 8599.871 ms 2024-05-14T17:09:26.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 281378 7.82 MB/sec execute 98 sec latency 9.697 ms 2024-05-14T17:09:27.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:26 smithi193 ceph-mon[35962]: pgmap v177: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 963 KiB/s wr, 104 op/s 2024-05-14T17:09:27.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:26 smithi150 ceph-mon[36204]: pgmap v177: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 963 KiB/s wr, 104 op/s 2024-05-14T17:09:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:26 smithi099 ceph-mon[31851]: pgmap v177: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 963 KiB/s wr, 104 op/s 2024-05-14T17:09:27.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 283924 7.90 MB/sec execute 99 sec latency 16.055 ms 2024-05-14T17:09:28.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 285452 7.89 MB/sec execute 100 sec latency 15.392 ms 2024-05-14T17:09:29.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:28 smithi193 ceph-mon[35962]: pgmap v178: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 963 KiB/s wr, 104 op/s 2024-05-14T17:09:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:28 smithi150 ceph-mon[36204]: pgmap v178: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 963 KiB/s wr, 104 op/s 2024-05-14T17:09:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:28 smithi099 ceph-mon[31851]: pgmap v178: 97 pgs: 97 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 963 KiB/s wr, 104 op/s 2024-05-14T17:09:29.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 287748 7.95 MB/sec execute 101 sec latency 16.012 ms 2024-05-14T17:09:30.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 289787 7.94 MB/sec execute 102 sec latency 17.180 ms 2024-05-14T17:09:31.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:30 smithi193 ceph-mon[35962]: pgmap v179: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-14T17:09:31.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:30 smithi150 ceph-mon[36204]: pgmap v179: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-14T17:09:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:30 smithi099 ceph-mon[31851]: pgmap v179: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-14T17:09:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 -- 172.21.15.99:0/2646932425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 msgr2=0x7fe02810f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 --2- 172.21.15.99:0/2646932425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02810f500 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ca0 tx=0x7fe02002a950 comp rx=0 tx=0).stop 2024-05-14T17:09:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 -- 172.21.15.99:0/2646932425 shutdown_connections 2024-05-14T17:09:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 --2- 172.21.15.99:0/2646932425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02810f500 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 --2- 172.21.15.99:0/2646932425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe028104620 0x7fe02810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 --2- 172.21.15.99:0/2646932425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe028103d00 0x7fe0281040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 -- 172.21.15.99:0/2646932425 >> 172.21.15.99:0/2646932425 conn(0x7fe028077cd0 msgr2=0x7fe0280780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 -- 172.21.15.99:0/2646932425 shutdown_connections 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.387+0000 7fe02dd72640 1 -- 172.21.15.99:0/2646932425 wait complete. 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 Processor -- start 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 -- start start 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe028103d00 0x7fe0281991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe028104620 0x7fe028199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe028112d10 con 0x7fe02810d0d0 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe028112b90 con 0x7fe028104620 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe02dd72640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe028112e90 con 0x7fe028103d00 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe027fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe027fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02819da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47924/0 (socket says 172.21.15.99:47924) 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe027fff640 1 -- 172.21.15.99:0/1617789109 learned_addr learned my addr 172.21.15.99:0/1617789109 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe0277fe640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe028103d00 0x7fe0281991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.388+0000 7fe026ffd640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe028104620 0x7fe028199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe027fff640 1 -- 172.21.15.99:0/1617789109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe028103d00 msgr2=0x7fe0281991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe027fff640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe028103d00 0x7fe0281991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe027fff640 1 -- 172.21.15.99:0/1617789109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe028104620 msgr2=0x7fe028199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe027fff640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe028104620 0x7fe028199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe027fff640 1 -- 172.21.15.99:0/1617789109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02819e1c0 con 0x7fe02810d0d0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe027fff640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02819da90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe020004af0 tx=0x7fe020031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe024ff9640 1 -- 172.21.15.99:0/1617789109 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020034040 con 0x7fe02810d0d0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe024ff9640 1 -- 172.21.15.99:0/1617789109 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0200308f0 con 0x7fe02810d0d0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe024ff9640 1 -- 172.21.15.99:0/1617789109 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0200045d0 con 0x7fe02810d0d0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02819e450 con 0x7fe02810d0d0 2024-05-14T17:09:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.389+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281a5d40 con 0x7fe02810d0d0 2024-05-14T17:09:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 Processor -- start 2024-05-14T17:09:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe024ff9640 1 -- 172.21.15.99:0/1617789109 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe020037020 con 0x7fe02810d0d0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 -- start start 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0fdee0 0x55c86f1002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c86f061c30 con 0x55c86f0ee7d0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c86f0517d0 con 0x55c86f0fdee0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe030a85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c86f055810 con 0x55c86f0ef0f0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.392+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0eebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47938/0 (socket says 172.21.15.99:47938) 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 -- 172.21.15.99:0/2457266520 learned_addr learned my addr 172.21.15.99:0/2457266520 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe0277fe640 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0fdee0 0x55c86f1002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 -- 172.21.15.99:0/2457266520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ef0f0 msgr2=0x55c86f0fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 -- 172.21.15.99:0/2457266520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0fdee0 msgr2=0x55c86f1002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0fdee0 0x55c86f1002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 -- 172.21.15.99:0/2457266520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86ed66210 con 0x55c86f0ee7d0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe024ff9640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0100b2a10 0x7fe0100b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe026ffd640 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0eebb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0137d0 tx=0x7fe01c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1871c018462e6b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe0277fe640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0100b2a10 0x7fe0100b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe024ff9640 1 -- 172.21.15.99:0/1617789109 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe020103120 con 0x7fe02810d0d0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe00dffb640 1 -- 172.21.15.99:0/2457266520 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c013d50 con 0x55c86f0ee7d0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe00dffb640 1 -- 172.21.15.99:0/2457266520 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe01c005020 con 0x55c86f0ee7d0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.393+0000 7fe00dffb640 1 -- 172.21.15.99:0/2457266520 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0044e0 con 0x55c86f0ee7d0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 -- 172.21.15.99:0/2457266520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 msgr2=0x55c86f0eebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe0277fe640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0100b2a10 0x7fe0100b4ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe018009da0 tx=0x7fe018005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0eebb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0137d0 tx=0x7fe01c005f90 comp rx=0 tx=0).stop 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 -- 172.21.15.99:0/2457266520 shutdown_connections 2024-05-14T17:09:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0fdee0 0x55c86f1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 --2- 172.21.15.99:0/2457266520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0eebb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 -- 172.21.15.99:0/2457266520 >> 172.21.15.99:0/2457266520 conn(0x55c86f0eae00 msgr2=0x55c86f0eb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 -- 172.21.15.99:0/2457266520 shutdown_connections 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.394+0000 7fe030a85740 1 -- 172.21.15.99:0/2457266520 wait complete. 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 -- start start 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 Processor -- start 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f0fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c86ed37790 con 0x55c86f0ee7d0 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c86f00d890 con 0x55c86f0ef0f0 2024-05-14T17:09:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c86f100bb0 con 0x55c86f0fdee0 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe030a85740 10 client.0 _ll_register_callbacks cb 0x55c86f010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe0277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe0277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47942/0 (socket says 172.21.15.99:47942) 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe0277fe640 1 -- 172.21.15.99:0/112651443 learned_addr learned my addr 172.21.15.99:0/112651443 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe027fff640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f0fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.395+0000 7fe026ffd640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe0277fe640 1 -- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 msgr2=0x55c86f0fb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe0277fe640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f0fb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe0277fe640 1 -- 172.21.15.99:0/112651443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 msgr2=0x55c86f0fab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe0277fe640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe0277fe640 1 -- 172.21.15.99:0/112651443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86ee32620 con 0x55c86f0ee7d0 2024-05-14T17:09:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe0277fe640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd890 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe0180045f0 tx=0x7fe018004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab7a0a09049f052 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe006ffd640 1 -- 172.21.15.99:0/112651443 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01801f640 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe01c005020 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe006ffd640 1 -- 172.21.15.99:0/112651443 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe01801f7e0 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe006ffd640 1 -- 172.21.15.99:0/112651443 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe01801f9e0 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe030a85740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c86f10c720 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.396+0000 7fe006ffd640 1 -- 172.21.15.99:0/112651443 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe01c005020 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe006ffd640 1 -- 172.21.15.99:0/112651443 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe018027070 con 0x55c86f0ee7d0 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe030a85740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe030a85740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe030a85740 20 client.15972 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe030a85740 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f10dca0 0x55c86f110090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe030a85740 4 client.15972 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86f063b80 con 0x55c86f10dca0 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe026ffd640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f10dca0 0x55c86f110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe026ffd640 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f10dca0 0x55c86f110090 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8638ea2bec8750a server_cookie=c4edff42b66e2b36 in_seq=0 out_seq=0 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.397+0000 7fe006ffd640 10 client.15972 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.398+0000 7fe006ffd640 1 -- 172.21.15.99:0/112651443 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c86f063b80 con 0x55c86f10dca0 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.398+0000 7fe006ffd640 10 client.15972 handle_command_reply: tid=0 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.398+0000 7fe030a85740 1 client.15972 shutdown 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 msgr2=0x55c86f0fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd890 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe0180045f0 tx=0x7fe018004730 comp rx=0 tx=0).stop 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 shutdown_connections 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f10dca0 0x55c86f110090 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f0fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f0fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 --2- 172.21.15.99:0/112651443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd890 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe006ffd640 0 client.15972 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 >> 172.21.15.99:0/112651443 conn(0x55c86f0eae00 msgr2=0x55c86f0f0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 shutdown_connections 2024-05-14T17:09:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 1 -- 172.21.15.99:0/112651443 wait complete. 2024-05-14T17:09:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.399+0000 7fe030a85740 20 client.15972 trim_cache size 0 max 16384 2024-05-14T17:09:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.414+0000 7fe030a85740 1 Processor -- start 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.414+0000 7fe030a85740 1 -- start start 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe030a85740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe030a85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 0x55c86f115960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe030a85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f115ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe030a85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aec40 con 0x55c86f0ef0f0 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe030a85740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281ac510 con 0x55c86f0ee7d0 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe030a85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aa360 con 0x55c86f0fdee0 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 0x55c86f115960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 0x55c86f115960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47948/0 (socket says 172.21.15.99:47948) 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 -- 172.21.15.99:0/800023079 learned_addr learned my addr 172.21.15.99:0/800023079 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe027fff640 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f115ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe0277fe640 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 -- 172.21.15.99:0/800023079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 msgr2=0x55c86f115ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f115ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 -- 172.21.15.99:0/800023079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ee7d0 msgr2=0x55c86f0fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 -- 172.21.15.99:0/800023079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86f0fb490 con 0x55c86f0ef0f0 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe026ffd640 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 0x55c86f115960 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe01c013770 tx=0x7fe01c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9199050ce3d37da6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe00d7fa640 1 -- 172.21.15.99:0/800023079 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c033070 con 0x55c86f0ef0f0 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe00d7fa640 1 -- 172.21.15.99:0/800023079 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe01c02de50 con 0x55c86f0ef0f0 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.415+0000 7fe00d7fa640 1 -- 172.21.15.99:0/800023079 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c02c6d0 con 0x55c86f0ef0f0 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 -- 172.21.15.99:0/800023079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 msgr2=0x55c86f115960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 0x55c86f115960 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe01c013770 tx=0x7fe01c009bd0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 -- 172.21.15.99:0/800023079 shutdown_connections 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0fdee0 0x55c86f115ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0ef0f0 0x55c86f115960 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 --2- 172.21.15.99:0/800023079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ee7d0 0x55c86f0fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 -- 172.21.15.99:0/800023079 >> 172.21.15.99:0/800023079 conn(0x55c86f0eae00 msgr2=0x55c86f0ea100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 -- 172.21.15.99:0/800023079 shutdown_connections 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.416+0000 7fe030a85740 1 -- 172.21.15.99:0/800023079 wait complete. 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 -- start start 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 Processor -- start 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ee7d0 0x55c86f101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 0x55c86f102730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aa9d0 con 0x55c86f0fdee0 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281abe40 con 0x55c86f0ef0f0 2024-05-14T17:09:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281ab0a0 con 0x55c86f0ee7d0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe030a85740 10 client.0 _ll_register_callbacks cb 0x7fe0180044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe027fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 0x55c86f102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe027fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 0x55c86f102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47950/0 (socket says 172.21.15.99:47950) 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe027fff640 1 -- 172.21.15.99:0/1846936617 learned_addr learned my addr 172.21.15.99:0/1846936617 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.417+0000 7fe0277fe640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ee7d0 0x55c86f101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe026ffd640 1 -- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ee7d0 msgr2=0x55c86f101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe026ffd640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ee7d0 0x55c86f101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe026ffd640 1 -- 172.21.15.99:0/1846936617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 msgr2=0x55c86f102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe026ffd640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 0x55c86f102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe026ffd640 1 -- 172.21.15.99:0/1846936617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86f1777b0 con 0x55c86f0ef0f0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe027fff640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 0x55c86f102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe026ffd640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f1021f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c86f0dce50 tx=0x7fe01c011f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1439b98b62dbcc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe006ffd640 1 -- 172.21.15.99:0/1846936617 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c033040 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe006ffd640 1 -- 172.21.15.99:0/1846936617 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fe01c005590 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe01c02de50 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe006ffd640 1 -- 172.21.15.99:0/1846936617 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe01c009a20 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe030a85740 10 client.25370 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c86f178ac0 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.418+0000 7fe006ffd640 1 -- 172.21.15.99:0/1846936617 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe01c02de50 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe006ffd640 1 -- 172.21.15.99:0/1846936617 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fe01c036070 con 0x55c86f0ef0f0 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe030a85740 10 client.25370 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe030a85740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe030a85740 20 client.25370 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe030a85740 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f178fd0 0x55c86f17b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe030a85740 4 client.25370 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c86f05d360 con 0x55c86f178fd0 2024-05-14T17:09:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe0277fe640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f178fd0 0x55c86f17b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe006ffd640 10 client.25370 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.419+0000 7fe0277fe640 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f178fd0 0x55c86f17b3c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45a712e4a12fdaa server_cookie=362b03412b1a304c in_seq=0 out_seq=0 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe006ffd640 1 -- 172.21.15.99:0/1846936617 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c86f05d360 con 0x55c86f178fd0 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe006ffd640 10 client.25370 handle_command_reply: tid=0 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 client.25370 shutdown 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 msgr2=0x55c86f1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f1021f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55c86f0dce50 tx=0x7fe01c011f40 comp rx=0 tx=0).stop 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 shutdown_connections 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c86f178fd0 0x55c86f17b3c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c86f0fdee0 0x55c86f102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c86f0ef0f0 0x55c86f1021f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe006ffd640 0 client.25370 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 --2- 172.21.15.99:0/1846936617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c86f0ee7d0 0x55c86f101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 >> 172.21.15.99:0/1846936617 conn(0x55c86f0eae00 msgr2=0x55c86f0f0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 shutdown_connections 2024-05-14T17:09:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 1 -- 172.21.15.99:0/1846936617 wait complete. 2024-05-14T17:09:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.457+0000 7fe030a85740 20 client.25370 trim_cache size 0 max 16384 2024-05-14T17:09:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:09:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:09:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0100b2a10 msgr2=0x7fe0100b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0100b2a10 0x7fe0100b4ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe018009da0 tx=0x7fe018005f90 comp rx=0 tx=0).stop 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 msgr2=0x7fe02819da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02819da90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe020004af0 tx=0x7fe020031f50 comp rx=0 tx=0).stop 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 shutdown_connections 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0100b2a10 0x7fe0100b4ed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe02810d0d0 0x7fe02819da90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe028104620 0x7fe028199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 --2- 172.21.15.99:0/1617789109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe028103d00 0x7fe0281991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 >> 172.21.15.99:0/1617789109 conn(0x7fe028077cd0 msgr2=0x7fe0280ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 shutdown_connections 2024-05-14T17:09:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:31.458+0000 7fe02dd72640 1 -- 172.21.15.99:0/1617789109 wait complete. 2024-05-14T17:09:31.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 291688 7.98 MB/sec execute 103 sec latency 48.193 ms 2024-05-14T17:09:32.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 293803 8.02 MB/sec execute 104 sec latency 29.422 ms 2024-05-14T17:09:33.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:32 smithi193 ceph-mon[35962]: pgmap v180: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 662 KiB/s wr, 84 op/s 2024-05-14T17:09:33.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:32 smithi150 ceph-mon[36204]: pgmap v180: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 662 KiB/s wr, 84 op/s 2024-05-14T17:09:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:32 smithi099 ceph-mon[31851]: pgmap v180: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 662 KiB/s wr, 84 op/s 2024-05-14T17:09:33.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 295826 8.06 MB/sec execute 105 sec latency 15.929 ms 2024-05-14T17:09:34.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 298254 8.11 MB/sec execute 106 sec latency 14.832 ms 2024-05-14T17:09:35.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:34 smithi193 ceph-mon[35962]: pgmap v181: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-05-14T17:09:35.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:34 smithi150 ceph-mon[36204]: pgmap v181: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-05-14T17:09:35.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:34 smithi099 ceph-mon[31851]: pgmap v181: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-05-14T17:09:35.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 300174 8.09 MB/sec execute 107 sec latency 31.033 ms 2024-05-14T17:09:36.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 302201 8.14 MB/sec execute 108 sec latency 12.568 ms 2024-05-14T17:09:37.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:36 smithi193 ceph-mon[35962]: pgmap v182: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-05-14T17:09:37.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:36 smithi150 ceph-mon[36204]: pgmap v182: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-05-14T17:09:37.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:36 smithi099 ceph-mon[31851]: pgmap v182: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-05-14T17:09:37.529 INFO:tasks.workunit.client.0.smithi099.stdout: 1 304534 8.18 MB/sec execute 109 sec latency 15.761 ms 2024-05-14T17:09:38.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 306521 8.21 MB/sec execute 110 sec latency 19.135 ms 2024-05-14T17:09:39.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:38 smithi193 ceph-mon[35962]: pgmap v183: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-05-14T17:09:39.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:38 smithi150 ceph-mon[36204]: pgmap v183: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-05-14T17:09:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:38 smithi099 ceph-mon[31851]: pgmap v183: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-05-14T17:09:39.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 308857 8.26 MB/sec execute 111 sec latency 16.572 ms 2024-05-14T17:09:40.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 310813 8.24 MB/sec execute 112 sec latency 16.304 ms 2024-05-14T17:09:41.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:40 smithi193 ceph-mon[35962]: pgmap v184: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-05-14T17:09:41.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:40 smithi150 ceph-mon[36204]: pgmap v184: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-05-14T17:09:41.263 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:40 smithi099 ceph-mon[31851]: pgmap v184: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-05-14T17:09:41.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 312872 8.29 MB/sec execute 113 sec latency 34.772 ms 2024-05-14T17:09:42.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 315135 8.33 MB/sec execute 114 sec latency 61.537 ms 2024-05-14T17:09:43.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:42 smithi193 ceph-mon[35962]: pgmap v185: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-05-14T17:09:43.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:42 smithi150 ceph-mon[36204]: pgmap v185: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-05-14T17:09:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:42 smithi099 ceph-mon[31851]: pgmap v185: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-05-14T17:09:43.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 317129 8.35 MB/sec execute 115 sec latency 18.160 ms 2024-05-14T17:09:44.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 319482 8.40 MB/sec execute 116 sec latency 14.976 ms 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 -- 172.21.15.99:0/4134406960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90104600 msgr2=0x7f6b9010cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/4134406960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90104600 0x7f6b9010cae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009900 tx=0x7f6b7c02a510 comp rx=0 tx=0).stop 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 -- 172.21.15.99:0/4134406960 shutdown_connections 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/4134406960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b9010d020 0x7f6b9010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/4134406960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90104600 0x7f6b9010cae0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/4134406960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b90103ce0 0x7f6b901040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 -- 172.21.15.99:0/4134406960 >> 172.21.15.99:0/4134406960 conn(0x7f6b90078ad0 msgr2=0x7f6b90078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 -- 172.21.15.99:0/4134406960 shutdown_connections 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.550+0000 7f6b96bc8640 1 -- 172.21.15.99:0/4134406960 wait complete. 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 Processor -- start 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 -- start start 2024-05-14T17:09:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 0x7f6b90199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b90104600 0x7f6b90199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b9010d020 0x7f6b9019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90112cd0 con 0x7f6b90103ce0 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90112b50 con 0x7f6b9010d020 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b96bc8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90112e50 con 0x7f6b90104600 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 0x7f6b90199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 0x7f6b90199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40266/0 (socket says 172.21.15.99:40266) 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 -- 172.21.15.99:0/158412494 learned_addr learned my addr 172.21.15.99:0/158412494 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b90104600 0x7f6b90199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b95165640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b9010d020 0x7f6b9019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 -- 172.21.15.99:0/158412494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b90104600 msgr2=0x7f6b90199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b90104600 0x7f6b90199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 -- 172.21.15.99:0/158412494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b9010d020 msgr2=0x7f6b9019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b9010d020 0x7f6b9019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 -- 172.21.15.99:0/158412494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9019e1b0 con 0x7f6b90103ce0 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.551+0000 7f6b94964640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 0x7f6b90199210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400e9f0 tx=0x7f6b8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.552+0000 7f6b8dffb640 1 -- 172.21.15.99:0/158412494 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400ccc0 con 0x7f6b90103ce0 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.552+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9019e4a0 con 0x7f6b90103ce0 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.552+0000 7f6b8dffb640 1 -- 172.21.15.99:0/158412494 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b840108e0 con 0x7f6b90103ce0 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.552+0000 7f6b8dffb640 1 -- 172.21.15.99:0/158412494 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84018880 con 0x7f6b90103ce0 2024-05-14T17:09:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.552+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b901a5d30 con 0x7f6b90103ce0 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.554+0000 7f6b8dffb640 1 -- 172.21.15.99:0/158412494 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6b84010430 con 0x7f6b90103ce0 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 Processor -- start 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 -- start start 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 0x5624969b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969c5ee0 0x5624969c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562496929c30 con 0x5624969c5ee0 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5624969197d0 con 0x5624969b7110 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b999c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56249691d810 con 0x5624969b67f0 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b95165640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 0x5624969b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b95165640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 0x5624969b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:42734/0 (socket says 172.21.15.99:42734) 2024-05-14T17:09:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b8ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969c5ee0 0x5624969c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b95165640 1 -- 172.21.15.99:0/1436660398 learned_addr learned my addr 172.21.15.99:0/1436660398 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b94964640 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.555+0000 7f6b8dffb640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b800b2a10 0x7f6b800b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 -- 172.21.15.99:0/1436660398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 msgr2=0x5624969c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 -- 172.21.15.99:0/1436660398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969c5ee0 msgr2=0x5624969c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969c5ee0 0x5624969c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 -- 172.21.15.99:0/1436660398 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56249662e210 con 0x5624969b67f0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969c5ee0 0x5624969c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b800b2a10 0x7f6b800b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b8dffb640 1 -- 172.21.15.99:0/158412494 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f6b840e29d0 con 0x7f6b90103ce0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b95165640 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 0x5624969b6bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b88011590 tx=0x7f6b88009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6171de22ef9789ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b7affd640 1 -- 172.21.15.99:0/1436660398 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88017050 con 0x5624969b67f0 2024-05-14T17:09:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b7affd640 1 -- 172.21.15.99:0/1436660398 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8801bbf0 con 0x5624969b67f0 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.556+0000 7f6b7affd640 1 -- 172.21.15.99:0/1436660398 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88026af0 con 0x5624969b67f0 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 -- 172.21.15.99:0/1436660398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 msgr2=0x5624969b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b95165640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b800b2a10 0x7f6b800b4ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b88023000 tx=0x7f6b88024000 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 0x5624969b6bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b88011590 tx=0x7f6b88009f90 comp rx=0 tx=0).stop 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 -- 172.21.15.99:0/1436660398 shutdown_connections 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969c5ee0 0x5624969c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1436660398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969b67f0 0x5624969b6bd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 -- 172.21.15.99:0/1436660398 >> 172.21.15.99:0/1436660398 conn(0x5624969b2e00 msgr2=0x5624969b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 -- 172.21.15.99:0/1436660398 shutdown_connections 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 -- 172.21.15.99:0/1436660398 wait complete. 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.557+0000 7f6b999c4740 1 -- start start 2024-05-14T17:09:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 Processor -- start 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b67f0 0x5624969c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 0x5624969c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624965ff790 con 0x5624969b7110 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5624968d5890 con 0x5624969b67f0 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5624969c8bb0 con 0x5624969c5ee0 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b8ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 0x5624969c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b999c4740 10 client.0 _ll_register_callbacks cb 0x5624968d8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b8ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 0x5624969c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40282/0 (socket says 172.21.15.99:40282) 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b8ffff640 1 -- 172.21.15.99:0/312081133 learned_addr learned my addr 172.21.15.99:0/312081133 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b94964640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b67f0 0x5624969c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.558+0000 7f6b95165640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b94964640 1 -- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 msgr2=0x5624969c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b94964640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b94964640 1 -- 172.21.15.99:0/312081133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 msgr2=0x5624969c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b94964640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 0x5624969c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b94964640 1 -- 172.21.15.99:0/312081133 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624966fa620 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 0x5624969c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b94964640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b67f0 0x5624969c57e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b840e0150 tx=0x7f6b84010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85337206d43a9d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b6ffff640 1 -- 172.21.15.99:0/312081133 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840aa3e0 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b6ffff640 1 -- 172.21.15.99:0/312081133 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f6b840a8f40 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8801bbf0 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b6ffff640 1 -- 172.21.15.99:0/312081133 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b840a9ce0 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b999c4740 10 client.25382 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624969d45f0 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.559+0000 7f6b6ffff640 1 -- 172.21.15.99:0/312081133 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8801bbf0 con 0x5624969b67f0 2024-05-14T17:09:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b6ffff640 1 -- 172.21.15.99:0/312081133 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6b840a8a40 con 0x5624969b67f0 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b999c4740 10 client.25382 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b999c4740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b999c4740 20 client.25382 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b999c4740 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5624969d5b00 0x5624969d7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b999c4740 4 client.25382 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56249692bb80 con 0x5624969d5b00 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5624969d5b00 0x5624969d7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5624969d5b00 0x5624969d7ef0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46f0c489b660070 server_cookie=be1dee6f45accaef in_seq=0 out_seq=0 2024-05-14T17:09:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.560+0000 7f6b6ffff640 10 client.25382 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.562+0000 7f6b6ffff640 1 -- 172.21.15.99:0/312081133 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56249692bb80 con 0x5624969d5b00 2024-05-14T17:09:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.562+0000 7f6b6ffff640 10 client.25382 handle_command_reply: tid=0 2024-05-14T17:09:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.562+0000 7f6b999c4740 1 client.25382 shutdown 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.562+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b67f0 msgr2=0x5624969c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.562+0000 7f6b999c4740 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b67f0 0x5624969c57e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b840e0150 tx=0x7f6b84010cf0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 shutdown_connections 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5624969d5b00 0x5624969d7ef0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b6ffff640 0 client.25382 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b7110 0x5624969c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 --2- 172.21.15.99:0/312081133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b67f0 0x5624969c57e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 >> 172.21.15.99:0/312081133 conn(0x5624969b2e00 msgr2=0x5624969b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 shutdown_connections 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 1 -- 172.21.15.99:0/312081133 wait complete. 2024-05-14T17:09:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.563+0000 7f6b999c4740 20 client.25382 trim_cache size 0 max 16384 2024-05-14T17:09:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 Processor -- start 2024-05-14T17:09:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 -- start start 2024-05-14T17:09:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901aec00 con 0x5624969b67f0 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901ac4d0 con 0x5624969b7110 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b999c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901aa320 con 0x5624969c5ee0 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40292/0 (socket says 172.21.15.99:40292) 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b95165640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 -- 172.21.15.99:0/605514415 learned_addr learned my addr 172.21.15.99:0/605514415 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 -- 172.21.15.99:0/605514415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 msgr2=0x5624969dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969dd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 -- 172.21.15.99:0/605514415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 msgr2=0x5624969c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.579+0000 7f6b94964640 1 -- 172.21.15.99:0/605514415 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624969b2c30 con 0x5624969b67f0 2024-05-14T17:09:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.580+0000 7f6b94964640 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969b1f80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b840a5d60 tx=0x7f6b840af220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b3fed14f6ff191 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.580+0000 7f6b7a7fc640 1 -- 172.21.15.99:0/605514415 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840b2b40 con 0x5624969b67f0 2024-05-14T17:09:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.580+0000 7f6b7a7fc640 1 -- 172.21.15.99:0/605514415 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84083070 con 0x5624969b67f0 2024-05-14T17:09:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.580+0000 7f6b7a7fc640 1 -- 172.21.15.99:0/605514415 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840a5650 con 0x5624969b67f0 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.580+0000 7f6b999c4740 1 -- 172.21.15.99:0/605514415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 msgr2=0x5624969b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.580+0000 7f6b999c4740 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969b1f80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b840a5d60 tx=0x7f6b840af220 comp rx=0 tx=0).stop 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- 172.21.15.99:0/605514415 shutdown_connections 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 --2- 172.21.15.99:0/605514415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969b1f80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- 172.21.15.99:0/605514415 >> 172.21.15.99:0/605514415 conn(0x5624969b2e00 msgr2=0x5624969b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- 172.21.15.99:0/605514415 shutdown_connections 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- 172.21.15.99:0/605514415 wait complete. 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- start start 2024-05-14T17:09:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 Processor -- start 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901aa990 con 0x5624969b67f0 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901abe00 con 0x5624969b7110 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901ab060 con 0x5624969c5ee0 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b999c4740 10 client.0 _ll_register_callbacks cb 0x7f6b840a96e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.581+0000 7f6b94964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40304/0 (socket says 172.21.15.99:40304) 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 -- 172.21.15.99:0/1898819526 learned_addr learned my addr 172.21.15.99:0/1898819526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b95165640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 -- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 msgr2=0x5624969c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 -- 172.21.15.99:0/1898819526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 msgr2=0x5624969c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 -- 172.21.15.99:0/1898819526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624969ca450 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b94964640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969ce370 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b840a9c70 tx=0x7f6b840caf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed704bdf194fa35 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b6ffff640 1 -- 172.21.15.99:0/1898819526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840aa3e0 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b6ffff640 1 -- 172.21.15.99:0/1898819526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f6b840af290 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b84083070 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b6ffff640 1 -- 172.21.15.99:0/1898819526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6b840b2b40 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b999c4740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624969ca450 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b6ffff640 1 -- 172.21.15.99:0/1898819526 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84083070 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b6ffff640 1 -- 172.21.15.99:0/1898819526 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6b840a5040 con 0x5624969b67f0 2024-05-14T17:09:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b999c4740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.582+0000 7f6b999c4740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b999c4740 20 client.15996 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562496a40630 0x562496a42a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b999c4740 4 client.15996 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562496925360 con 0x562496a40630 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562496a40630 0x562496a42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b8ffff640 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562496a40630 0x562496a42a20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4a88a49b7b622d server_cookie=d7cc7c3e04b311c4 in_seq=0 out_seq=0 2024-05-14T17:09:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.583+0000 7f6b6ffff640 10 client.15996 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b6ffff640 1 -- 172.21.15.99:0/1898819526 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562496925360 con 0x562496a40630 2024-05-14T17:09:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b6ffff640 10 client.15996 handle_command_reply: tid=0 2024-05-14T17:09:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 client.15996 shutdown 2024-05-14T17:09:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 msgr2=0x5624969ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969ce370 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b840a9c70 tx=0x7f6b840caf90 comp rx=0 tx=0).stop 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 shutdown_connections 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562496a40630 0x562496a42a20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624969c5ee0 0x5624969c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624969b7110 0x5624969c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b6ffff640 0 client.15996 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 --2- 172.21.15.99:0/1898819526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624969b67f0 0x5624969ce370 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 >> 172.21.15.99:0/1898819526 conn(0x5624969b2e00 msgr2=0x5624969b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 shutdown_connections 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 1 -- 172.21.15.99:0/1898819526 wait complete. 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.644+0000 7f6b999c4740 20 client.15996 trim_cache size 0 max 16384 2024-05-14T17:09:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:09:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:09:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:09:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b800b2a10 msgr2=0x7f6b800b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b800b2a10 0x7f6b800b4ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b88023000 tx=0x7f6b88024000 comp rx=0 tx=0).stop 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 msgr2=0x7f6b90199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 0x7f6b90199210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400e9f0 tx=0x7f6b8400edf0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 shutdown_connections 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b800b2a10 0x7f6b800b4ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b9010d020 0x7f6b9019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b90104600 0x7f6b90199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 --2- 172.21.15.99:0/158412494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b90103ce0 0x7f6b90199210 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.645+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 >> 172.21.15.99:0/158412494 conn(0x7f6b90078ad0 msgr2=0x7f6b900fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.646+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 shutdown_connections 2024-05-14T17:09:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:44.646+0000 7f6b96bc8640 1 -- 172.21.15.99:0/158412494 wait complete. 2024-05-14T17:09:45.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:44 smithi193 ceph-mon[35962]: pgmap v186: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-14T17:09:45.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:44 smithi150 ceph-mon[36204]: pgmap v186: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-14T17:09:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:44 smithi099 ceph-mon[31851]: pgmap v186: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-14T17:09:45.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 321072 8.38 MB/sec execute 117 sec latency 28.668 ms 2024-05-14T17:09:46.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 323349 8.42 MB/sec execute 118 sec latency 11.165 ms 2024-05-14T17:09:47.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:46 smithi193 ceph-mon[35962]: pgmap v187: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T17:09:47.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:46 smithi150 ceph-mon[36204]: pgmap v187: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T17:09:47.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:46 smithi099 ceph-mon[31851]: pgmap v187: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T17:09:47.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 325133 8.41 MB/sec execute 119 sec latency 100.833 ms 2024-05-14T17:09:48.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 327014 8.44 MB/sec execute 120 sec latency 19.456 ms 2024-05-14T17:09:49.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:48 smithi193 ceph-mon[35962]: pgmap v188: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T17:09:49.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:48 smithi099 ceph-mon[31851]: pgmap v188: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T17:09:49.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:48 smithi150 ceph-mon[36204]: pgmap v188: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-14T17:09:49.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 329216 8.47 MB/sec execute 121 sec latency 14.573 ms 2024-05-14T17:09:50.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331009 8.48 MB/sec execute 122 sec latency 17.377 ms 2024-05-14T17:09:51.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:50 smithi150 ceph-mon[36204]: pgmap v189: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-05-14T17:09:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:50 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:09:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:50 smithi099 ceph-mon[31851]: pgmap v189: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-05-14T17:09:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:50 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:09:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:50 smithi193 ceph-mon[35962]: pgmap v189: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 377 op/s 2024-05-14T17:09:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:50 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:09:51.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.44 MB/sec execute 123 sec latency 628.058 ms 2024-05-14T17:09:52.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.37 MB/sec execute 124 sec latency 1628.111 ms 2024-05-14T17:09:53.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:52 smithi150 ceph-mon[36204]: pgmap v190: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-05-14T17:09:53.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:52 smithi099 ceph-mon[31851]: pgmap v190: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-05-14T17:09:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:52 smithi193 ceph-mon[35962]: pgmap v190: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-05-14T17:09:53.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.30 MB/sec execute 125 sec latency 2628.159 ms 2024-05-14T17:09:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:09:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:09:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:09:54.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.23 MB/sec execute 126 sec latency 3628.255 ms 2024-05-14T17:09:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:54 smithi150 ceph-mon[36204]: pgmap v191: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-05-14T17:09:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:54 smithi099 ceph-mon[31851]: pgmap v191: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-05-14T17:09:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:54 smithi193 ceph-mon[35962]: pgmap v191: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-05-14T17:09:55.530 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.17 MB/sec execute 127 sec latency 4628.298 ms 2024-05-14T17:09:56.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.11 MB/sec execute 128 sec latency 5628.396 ms 2024-05-14T17:09:57.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:56 smithi150 ceph-mon[36204]: pgmap v192: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-05-14T17:09:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:56 smithi099 ceph-mon[31851]: pgmap v192: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-05-14T17:09:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:56 smithi193 ceph-mon[35962]: pgmap v192: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-05-14T17:09:57.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 8.04 MB/sec execute 129 sec latency 6628.492 ms 2024-05-14T17:09:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 -- 172.21.15.99:0/3753858754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84108a70 msgr2=0x7fdf84108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 --2- 172.21.15.99:0/3753858754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84108a70 0x7fdf84108e50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009cd0 tx=0x7fdf8002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 -- 172.21.15.99:0/3753858754 shutdown_connections 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 --2- 172.21.15.99:0/3753858754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84103420 0x7fdf8410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 --2- 172.21.15.99:0/3753858754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 0x7fdf84102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 --2- 172.21.15.99:0/3753858754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84108a70 0x7fdf84108e50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 -- 172.21.15.99:0/3753858754 >> 172.21.15.99:0/3753858754 conn(0x7fdf84079ee0 msgr2=0x7fdf8407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 -- 172.21.15.99:0/3753858754 shutdown_connections 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 -- 172.21.15.99:0/3753858754 wait complete. 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.744+0000 7fdf8b659640 1 Processor -- start 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 -- start start 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 0x7fdf841991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 0x7fdf84199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84108a70 0x7fdf8419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84112d80 con 0x7fdf84103420 2024-05-14T17:09:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84112c00 con 0x7fdf84102a60 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf8b659640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84112f00 con 0x7fdf84108a70 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 0x7fdf84199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 0x7fdf84199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39128/0 (socket says 172.21.15.99:39128) 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2627558147 learned_addr learned my addr 172.21.15.99:0/2627558147 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84108a70 0x7fdf8419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf893f5640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 0x7fdf841991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2627558147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84108a70 msgr2=0x7fdf8419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84108a70 0x7fdf8419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2627558147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 msgr2=0x7fdf841991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 0x7fdf841991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2627558147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8419e180 con 0x7fdf84103420 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf893f5640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 0x7fdf841991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 0x7fdf84199720 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400d950 tx=0x7fdf7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.745+0000 7fdf7a7fc640 1 -- 172.21.15.99:0/2627558147 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74017070 con 0x7fdf84103420 2024-05-14T17:09:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.746+0000 7fdf7a7fc640 1 -- 172.21.15.99:0/2627558147 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf74009c30 con 0x7fdf84103420 2024-05-14T17:09:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.746+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8419e470 con 0x7fdf84103420 2024-05-14T17:09:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.746+0000 7fdf7a7fc640 1 -- 172.21.15.99:0/2627558147 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400f620 con 0x7fdf84103420 2024-05-14T17:09:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.746+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf841a5d00 con 0x7fdf84103420 2024-05-14T17:09:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.748+0000 7fdf7a7fc640 1 -- 172.21.15.99:0/2627558147 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdf74010460 con 0x7fdf84103420 2024-05-14T17:09:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.748+0000 7fdf8e379740 1 Processor -- start 2024-05-14T17:09:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.748+0000 7fdf8e379740 1 -- start start 2024-05-14T17:09:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8474048f0 0x55d847404cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847411ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf8e379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d847377c30 con 0x55d847405210 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf8e379740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8473677d0 con 0x55d8474048f0 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf8e379740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d84736b810 con 0x55d84740fad0 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39130/0 (socket says 172.21.15.99:39130) 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 -- 172.21.15.99:0/3311031341 learned_addr learned my addr 172.21.15.99:0/3311031341 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8474048f0 0x55d847404cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847411ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 -- 172.21.15.99:0/3311031341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 msgr2=0x55d847411ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847411ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 -- 172.21.15.99:0/3311031341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8474048f0 msgr2=0x55d847404cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8474048f0 0x55d847404cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 -- 172.21.15.99:0/3311031341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84707c210 con 0x55d847405210 2024-05-14T17:09:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847411ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf7a7fc640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf700b2a10 0x7fdf700b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf893f5640 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740f590 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009cd0 tx=0x7fdf80005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631a868ab885716f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf700b2a10 0x7fdf700b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.749+0000 7fdf7a7fc640 1 -- 172.21.15.99:0/2627558147 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdf740e2220 con 0x7fdf84103420 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf6f7fe640 1 -- 172.21.15.99:0/3311031341 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80031a50 con 0x55d847405210 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf6f7fe640 1 -- 172.21.15.99:0/3311031341 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80036040 con 0x55d847405210 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf6f7fe640 1 -- 172.21.15.99:0/3311031341 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf800045e0 con 0x55d847405210 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 -- 172.21.15.99:0/3311031341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 msgr2=0x55d84740f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf700b2a10 0x7fdf700b4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c005fd0 tx=0x7fdf7c00b260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740f590 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009cd0 tx=0x7fdf80005f90 comp rx=0 tx=0).stop 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 -- 172.21.15.99:0/3311031341 shutdown_connections 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847411ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740f590 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3311031341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8474048f0 0x55d847404cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 -- 172.21.15.99:0/3311031341 >> 172.21.15.99:0/3311031341 conn(0x55d847400e00 msgr2=0x55d8474011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 -- 172.21.15.99:0/3311031341 shutdown_connections 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.750+0000 7fdf8e379740 1 -- 172.21.15.99:0/3311031341 wait complete. 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 -- start start 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 Processor -- start 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d84740f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d84740cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d84704d790 con 0x55d847405210 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d847323890 con 0x55d84740fad0 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d847412740 con 0x55d8474048f0 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf8e379740 10 client.0 _ll_register_callbacks cb 0x55d847326d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.751+0000 7fdf88bf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39138/0 (socket says 172.21.15.99:39138) 2024-05-14T17:09:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2511085608 learned_addr learned my addr 172.21.15.99:0/2511085608 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d84740cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf893f5640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d84740f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 msgr2=0x55d84740f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d84740f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2511085608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 msgr2=0x55d84740cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d84740cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 -- 172.21.15.99:0/2511085608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d847148620 con 0x55d847405210 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d84740cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740c6d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdf740a9230 tx=0x7fdf740a9430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473a8e200da8c4e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf6cff9640 1 -- 172.21.15.99:0/2511085608 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74020050 con 0x55d847405210 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf6cff9640 1 -- 172.21.15.99:0/2511085608 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdf740e5c40 con 0x55d847405210 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf80036040 con 0x55d847405210 2024-05-14T17:09:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf6cff9640 1 -- 172.21.15.99:0/2511085608 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf740910e0 con 0x55d847405210 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf8e379740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d84741e110 con 0x55d847405210 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf6cff9640 1 -- 172.21.15.99:0/2511085608 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80036040 con 0x55d847405210 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.752+0000 7fdf6cff9640 1 -- 172.21.15.99:0/2511085608 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fdf740a4ea0 con 0x55d847405210 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf8e379740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf8e379740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf8e379740 20 client.16008 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf8e379740 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84741f620 0x55d847421a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf8e379740 4 client.16008 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d847379b80 con 0x55d84741f620 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf893f5640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84741f620 0x55d847421a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf893f5640 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84741f620 0x55d847421a30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8fe429cd5c315d server_cookie=e6c65fb9b85756f9 in_seq=0 out_seq=0 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.753+0000 7fdf6cff9640 10 client.16008 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf6cff9640 1 -- 172.21.15.99:0/2511085608 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d847379b80 con 0x55d84741f620 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf6cff9640 10 client.16008 handle_command_reply: tid=0 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 client.16008 shutdown 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 msgr2=0x55d84740c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740c6d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdf740a9230 tx=0x7fdf740a9430 comp rx=0 tx=0).stop 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 shutdown_connections 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84741f620 0x55d847421a30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d84740cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d84740c6d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf6cff9640 0 client.16008 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 --2- 172.21.15.99:0/2511085608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d84740f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 >> 172.21.15.99:0/2511085608 conn(0x55d847400e00 msgr2=0x55d847400100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 shutdown_connections 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 1 -- 172.21.15.99:0/2511085608 wait complete. 2024-05-14T17:09:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.754+0000 7fdf8e379740 20 client.16008 trim_cache size 0 max 16384 2024-05-14T17:09:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 Processor -- start 2024-05-14T17:09:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 -- start start 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 0x55d84741b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d847405210 0x55d84740ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847427340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841b0de0 con 0x55d8474048f0 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841ae680 con 0x55d847405210 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf8e379740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841ac4d0 con 0x55d84740fad0 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 0x55d84741b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 0x55d84741b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39142/0 (socket says 172.21.15.99:39142) 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 -- 172.21.15.99:0/1246597863 learned_addr learned my addr 172.21.15.99:0/1246597863 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d847405210 0x55d84740ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 -- 172.21.15.99:0/1246597863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 msgr2=0x55d847427340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847427340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 -- 172.21.15.99:0/1246597863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d847405210 msgr2=0x55d84740ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d847405210 0x55d84740ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 -- 172.21.15.99:0/1246597863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d847400c30 con 0x55d8474048f0 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf893f5640 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 0x55d84741b610 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdf80004230 tx=0x7fdf800041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac341ec072e2581 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf6effd640 1 -- 172.21.15.99:0/1246597863 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80057070 con 0x55d8474048f0 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf6effd640 1 -- 172.21.15.99:0/1246597863 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80036050 con 0x55d8474048f0 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.770+0000 7fdf6effd640 1 -- 172.21.15.99:0/1246597863 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80003bd0 con 0x55d8474048f0 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 -- 172.21.15.99:0/1246597863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 msgr2=0x55d84741b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 0x55d84741b610 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdf80004230 tx=0x7fdf800041c0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 -- 172.21.15.99:0/1246597863 shutdown_connections 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d84740fad0 0x55d847427340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d847405210 0x55d84740ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 --2- 172.21.15.99:0/1246597863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8474048f0 0x55d84741b610 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 -- 172.21.15.99:0/1246597863 >> 172.21.15.99:0/1246597863 conn(0x55d847400e00 msgr2=0x55d847406b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 -- 172.21.15.99:0/1246597863 shutdown_connections 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 -- 172.21.15.99:0/1246597863 wait complete. 2024-05-14T17:09:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 -- start start 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.771+0000 7fdf8e379740 1 Processor -- start 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d847413670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d847413bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d8474140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841acb40 con 0x55d847405210 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841adfb0 con 0x55d84740fad0 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841ad210 con 0x55d8474048f0 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf8e379740 10 client.0 _ll_register_callbacks cb 0x7fdf7400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d847413bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d847413bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39158/0 (socket says 172.21.15.99:39158) 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 -- 172.21.15.99:0/3950336257 learned_addr learned my addr 172.21.15.99:0/3950336257 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d8474140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf893f5640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d847413670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 -- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 msgr2=0x55d847413670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d847413670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 -- 172.21.15.99:0/3950336257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 msgr2=0x55d8474140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d8474140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 -- 172.21.15.99:0/3950336257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d847489020 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf89bf6640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d8474140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf88bf4640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d847413bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf740dfa50 tx=0x7fdf740a89c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2c9e1e57d2c5ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.772+0000 7fdf6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf6cff9640 1 -- 172.21.15.99:0/3950336257 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74020070 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf6cff9640 1 -- 172.21.15.99:0/3950336257 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7fdf74010960 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf80036050 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf6cff9640 1 -- 172.21.15.99:0/3950336257 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf740a8ab0 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d84748a310 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf6cff9640 1 -- 172.21.15.99:0/3950336257 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80036050 con 0x55d847405210 2024-05-14T17:09:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf6cff9640 1 -- 172.21.15.99:0/3950336257 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fdf740ca300 con 0x55d847405210 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 20 client.16014 populate_metadata read hostname 'smithi099' 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84748a890 0x55d84748ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 4 client.16014 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.773+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d847373360 con 0x55d84748a890 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.774+0000 7fdf893f5640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84748a890 0x55d84748ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.774+0000 7fdf6cff9640 10 client.16014 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.774+0000 7fdf893f5640 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84748a890 0x55d84748ccb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96e844907a65316 server_cookie=26646bf4fc463e80 in_seq=0 out_seq=0 2024-05-14T17:09:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf6cff9640 1 -- 172.21.15.99:0/3950336257 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d847373360 con 0x55d84748a890 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf6cff9640 10 client.16014 handle_command_reply: tid=0 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 client.16014 shutdown 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 msgr2=0x55d847413bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d847413bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdf740dfa50 tx=0x7fdf740a89c0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 shutdown_connections 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d84748a890 0x55d84748ccb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf6cff9640 0 client.16014 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d84740fad0 0x55d8474140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d847405210 0x55d847413bb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 --2- 172.21.15.99:0/3950336257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8474048f0 0x55d847413670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 >> 172.21.15.99:0/3950336257 conn(0x55d847400e00 msgr2=0x55d847407350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 shutdown_connections 2024-05-14T17:09:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 1 -- 172.21.15.99:0/3950336257 wait complete. 2024-05-14T17:09:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.814+0000 7fdf8e379740 20 client.16014 trim_cache size 0 max 16384 2024-05-14T17:09:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:09:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:09:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf700b2a10 msgr2=0x7fdf700b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf700b2a10 0x7fdf700b4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c005fd0 tx=0x7fdf7c00b260 comp rx=0 tx=0).stop 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 msgr2=0x7fdf84199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 0x7fdf84199720 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400d950 tx=0x7fdf7400de10 comp rx=0 tx=0).stop 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 shutdown_connections 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf700b2a10 0x7fdf700b4ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84108a70 0x7fdf8419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf84103420 0x7fdf84199720 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 --2- 172.21.15.99:0/2627558147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf84102a60 0x7fdf841991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 >> 172.21.15.99:0/2627558147 conn(0x7fdf84079ee0 msgr2=0x7fdf841002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 shutdown_connections 2024-05-14T17:09:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:09:57.815+0000 7fdf8b659640 1 -- 172.21.15.99:0/2627558147 wait complete. 2024-05-14T17:09:58.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 7.98 MB/sec execute 130 sec latency 7628.604 ms 2024-05-14T17:09:59.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:09:58 smithi150 ceph-mon[36204]: pgmap v193: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-05-14T17:09:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:09:58 smithi099 ceph-mon[31851]: pgmap v193: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-05-14T17:09:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:09:58 smithi193 ceph-mon[35962]: pgmap v193: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-05-14T17:09:59.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 331731 7.92 MB/sec execute 131 sec latency 8628.738 ms 2024-05-14T17:09:59.674 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.7 2024-05-14T17:09:59.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:09:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:10:00.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 333824 7.96 MB/sec execute 132 sec latency 8688.903 ms 2024-05-14T17:10:01.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:00 smithi099 ceph-mon[31851]: pgmap v194: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-05-14T17:10:01.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:00 smithi099 ceph-mon[31851]: overall HEALTH_OK 2024-05-14T17:10:01.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:00 smithi150 ceph-mon[36204]: pgmap v194: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-05-14T17:10:01.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:00 smithi150 ceph-mon[36204]: overall HEALTH_OK 2024-05-14T17:10:01.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:00 smithi193 ceph-mon[35962]: pgmap v194: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-05-14T17:10:01.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:00 smithi193 ceph-mon[35962]: overall HEALTH_OK 2024-05-14T17:10:01.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335037 7.94 MB/sec execute 133 sec latency 30.176 ms 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 -- 172.21.15.99:0/1233528119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01037f0 msgr2=0x7f9ee0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/1233528119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01037f0 0x7f9ee0103bd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00099c0 tx=0x7f9ed0009cd0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 -- 172.21.15.99:0/1233528119 shutdown_connections 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/1233528119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee01088f0 0x7f9ee0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/1233528119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01037f0 0x7f9ee0103bd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/1233528119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee01041a0 0x7f9ee01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 -- 172.21.15.99:0/1233528119 >> 172.21.15.99:0/1233528119 conn(0x7f9ee007a6a0 msgr2=0x7f9ee007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 -- 172.21.15.99:0/1233528119 shutdown_connections 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.769+0000 7f9ee4c42700 1 -- 172.21.15.99:0/1233528119 wait complete. 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 Processor -- start 2024-05-14T17:10:01.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 -- start start 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee01037f0 0x7f9ee019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee01041a0 0x7f9ee019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 0x7f9ee0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01136e0 con 0x7f9ee01088f0 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0113560 con 0x7f9ee01037f0 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9ee4c42700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0113860 con 0x7f9ee01041a0 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9eded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 0x7f9ee0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9eded9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 0x7f9ee0199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39164/0 (socket says 172.21.15.99:39164) 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.770+0000 7f9eded9d700 1 -- 172.21.15.99:0/621712297 learned_addr learned my addr 172.21.15.99:0/621712297 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9ede59c700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee01037f0 0x7f9ee019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eddd9b700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee01041a0 0x7f9ee019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eded9d700 1 -- 172.21.15.99:0/621712297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee01041a0 msgr2=0x7f9ee019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:01.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eded9d700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee01041a0 0x7f9ee019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eded9d700 1 -- 172.21.15.99:0/621712297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee01037f0 msgr2=0x7f9ee019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eded9d700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee01037f0 0x7f9ee019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eded9d700 1 -- 172.21.15.99:0/621712297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee019a180 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9eded9d700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 0x7f9ee0199a20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800ada0 tx=0x7f9ed800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8017020 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed800cd80 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8005260 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee019a470 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.771+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee01a6730 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.772+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee010bb30 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.773+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9ed800c8d0 con 0x7f9ee01088f0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.774+0000 7f9ecf7fe700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9ec80b29f0 0x7f9ec80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.774+0000 7f9ede59c700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9ec80b29f0 0x7f9ec80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.774+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f9ed80e1cf0 con 0x7f9ee01088f0 2024-05-14T17:10:01.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.774+0000 7f9ede59c700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9ec80b29f0 0x7f9ec80b4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0005fb0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:01.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:01.778+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ed80e6070 con 0x7f9ee01088f0 2024-05-14T17:10:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.050+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ee010b510 con 0x7f9ee01088f0 2024-05-14T17:10:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.051+0000 7f9ecf7fe700 1 -- 172.21.15.99:0/621712297 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11877 (secure 0 0 0) 0x7f9ed8021020 con 0x7f9ee01088f0 2024-05-14T17:10:02.051 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:10:02.051 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:09:49.585995+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9ec80b29f0 msgr2=0x7f9ec80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9ec80b29f0 0x7f9ec80b4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0005fb0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 msgr2=0x7f9ee0199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 0x7f9ee0199a20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800ada0 tx=0x7f9ed800c330 comp rx=0 tx=0).stop 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 shutdown_connections 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9ec80b29f0 0x7f9ec80b4eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ee01088f0 0x7f9ee0199a20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee01037f0 0x7f9ee019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 --2- 172.21.15.99:0/621712297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee01041a0 0x7f9ee019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.053+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 >> 172.21.15.99:0/621712297 conn(0x7f9ee007a6a0 msgr2=0x7f9ee0109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.054+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 shutdown_connections 2024-05-14T17:10:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:02.054+0000 7f9ee4c42700 1 -- 172.21.15.99:0/621712297 wait complete. 2024-05-14T17:10:02.054 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 28 2024-05-14T17:10:02.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335120 7.88 MB/sec execute 134 sec latency 946.817 ms 2024-05-14T17:10:02.856 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 133.7 --await-for 45.0' 2024-05-14T17:10:02.857 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 133.7 --await-for 45.0 2024-05-14T17:10:03.105 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:10:03.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:02 smithi150 ceph-mon[36204]: pgmap v195: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 608 KiB/s wr, 76 op/s 2024-05-14T17:10:03.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:02 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/621712297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:03.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:02 smithi099 ceph-mon[31851]: pgmap v195: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 608 KiB/s wr, 76 op/s 2024-05-14T17:10:03.205 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:02 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/621712297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:02 smithi193 ceph-mon[35962]: pgmap v195: 97 pgs: 97 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 608 KiB/s wr, 76 op/s 2024-05-14T17:10:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:02 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/621712297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:03.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335120 7.83 MB/sec execute 135 sec latency 1946.843 ms 2024-05-14T17:10:04.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335120 7.77 MB/sec execute 136 sec latency 2947.148 ms 2024-05-14T17:10:04.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 -- 172.21.15.99:0/538361968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770108840 msgr2=0x7f7770110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 --2- 172.21.15.99:0/538361968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770108840 0x7f7770110450 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f776800b0d0 tx=0x7f776800b3e0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 -- 172.21.15.99:0/538361968 shutdown_connections 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 --2- 172.21.15.99:0/538361968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770108840 0x7f7770110450 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 --2- 172.21.15.99:0/538361968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7770103850 0x7f7770103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 --2- 172.21.15.99:0/538361968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770104200 0x7f7770108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 -- 172.21.15.99:0/538361968 >> 172.21.15.99:0/538361968 conn(0x7f777007a610 msgr2=0x7f777007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 -- 172.21.15.99:0/538361968 shutdown_connections 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.736+0000 7f777643f700 1 -- 172.21.15.99:0/538361968 wait complete. 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 Processor -- start 2024-05-14T17:10:04.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 -- start start 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770103850 0x7f777019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 0x7f777019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7770108840 0x7f77701a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7770113730 con 0x7f7770104200 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77701135b0 con 0x7f7770103850 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f777643f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77701138b0 con 0x7f7770108840 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 0x7f777019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 0x7f777019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43094/0 (socket says 172.21.15.99:43094) 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f7767fff700 1 -- 172.21.15.99:0/3194906495 learned_addr learned my addr 172.21.15.99:0/3194906495 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f776ffff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770103850 0x7f777019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.737+0000 7f77749dc700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7770108840 0x7f77701a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f7767fff700 1 -- 172.21.15.99:0/3194906495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7770108840 msgr2=0x7f77701a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f7767fff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7770108840 0x7f77701a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f7767fff700 1 -- 172.21.15.99:0/3194906495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770103850 msgr2=0x7f777019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f7767fff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770103850 0x7f777019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f7767fff700 1 -- 172.21.15.99:0/3194906495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77701a0de0 con 0x7f7770104200 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f776ffff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770103850 0x7f777019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f7767fff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 0x7f777019c310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f775800d980 tx=0x7f775800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f776dffb700 1 -- 172.21.15.99:0/3194906495 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758017070 con 0x7f7770104200 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f776dffb700 1 -- 172.21.15.99:0/3194906495 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f775800fd40 con 0x7f7770104200 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f776dffb700 1 -- 172.21.15.99:0/3194906495 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775800e3b0 con 0x7f7770104200 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77701a10d0 con 0x7f7770104200 2024-05-14T17:10:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.738+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77701a8950 con 0x7f7770104200 2024-05-14T17:10:04.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.741+0000 7f776dffb700 1 -- 172.21.15.99:0/3194906495 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7758010610 con 0x7f7770104200 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.741+0000 7f776dffb700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f775c0b2b20 0x7f775c0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.741+0000 7f776ffff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f775c0b2b20 0x7f775c0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f776dffb700 1 -- 172.21.15.99:0/3194906495 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f77580e1f50 con 0x7f7770104200 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 Processor -- start 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f776ffff700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f775c0b2b20 0x7f775c0b4fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7760011490 tx=0x7f7760009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 -- start start 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d6cd023f00 con 0x55d6cd0cadd0 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d6cd013aa0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6cd017ae0 con 0x55d6cd0c6350 2024-05-14T17:10:04.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43108/0 (socket says 172.21.15.99:43108) 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f7767fff700 1 -- 172.21.15.99:0/1046167397 learned_addr learned my addr 172.21.15.99:0/1046167397 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f776ffff700 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.742+0000 7f77749dc700 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f7767fff700 1 -- 172.21.15.99:0/1046167397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 msgr2=0x55d6cd0c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f7767fff700 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f7767fff700 1 -- 172.21.15.99:0/1046167397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0ca4b0 msgr2=0x55d6cd0ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f7767fff700 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f7767fff700 1 -- 172.21.15.99:0/1046167397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ccfbd140 con 0x55d6cd0cadd0 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f776ffff700 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f7767fff700 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c5d30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f77580a91f0 tx=0x7f77580a8aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307d9b874ae600b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f77667fc700 1 -- 172.21.15.99:0/1046167397 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758020040 con 0x55d6cd0cadd0 2024-05-14T17:10:04.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f77667fc700 1 -- 172.21.15.99:0/1046167397 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77580a9260 con 0x55d6cd0cadd0 2024-05-14T17:10:04.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.743+0000 7f77667fc700 1 -- 172.21.15.99:0/1046167397 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580ad120 con 0x55d6cd0cadd0 2024-05-14T17:10:04.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1046167397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 msgr2=0x55d6cd0c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c5d30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f77580a91f0 tx=0x7f77580a8aa0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1046167397 shutdown_connections 2024-05-14T17:10:04.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1046167397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c5d30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1046167397 >> 172.21.15.99:0/1046167397 conn(0x55d6ccff2f40 msgr2=0x55d6cd0c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1046167397 shutdown_connections 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1046167397 wait complete. 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 -- start start 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.744+0000 7f7780cd5b80 1 Processor -- start 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d6ccf1a130 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d6ccf7bf30 con 0x55d6cd0c6350 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6cd0d2850 con 0x55d6cd0cadd0 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43116/0 (socket says 172.21.15.99:43116) 2024-05-14T17:10:04.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 -- 172.21.15.99:0/1077284587 learned_addr learned my addr 172.21.15.99:0/1077284587 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7780cd5b80 10 client.0 _ll_register_callbacks cb 0x55d6ccfaeae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f776ffff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f77749dc700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 -- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 msgr2=0x55d6cd0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 -- 172.21.15.99:0/1077284587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 msgr2=0x55d6cd0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 -- 172.21.15.99:0/1077284587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ccf63a60 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f776ffff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f7767fff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0cefb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f77580e2270 tx=0x7f7758010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3664d26d50d3973f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.745+0000 7f774f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f774f7fe700 1 -- 172.21.15.99:0/1077284587 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580c2020 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77580a9260 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f774f7fe700 1 -- 172.21.15.99:0/1077284587 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f77580b1e40 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f774f7fe700 1 -- 172.21.15.99:0/1077284587 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77580b20c0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 10 client.16032 fetch_fsmap learned FSMap version 28 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6ccfcd8f0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f774f7fe700 1 -- 172.21.15.99:0/1077284587 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77580a9260 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f774f7fe700 1 -- 172.21.15.99:0/1077284587 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f77580a4bd0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 10 client.16032 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 20 client.16032 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd0df2b0 0x55d6cd0e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 4 client.16032 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.746+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6cd025e50 con 0x55d6cd0df2b0 2024-05-14T17:10:04.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.747+0000 7f776ffff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd0df2b0 0x55d6cd0e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.747+0000 7f776ffff700 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd0df2b0 0x55d6cd0e16a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90ca76ef75a04af server_cookie=d8533e892114a3a in_seq=0 out_seq=0 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.747+0000 7f774f7fe700 10 client.16032 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f774f7fe700 1 -- 172.21.15.99:0/1077284587 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6cd025e50 con 0x55d6cd0df2b0 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f774f7fe700 10 client.16032 handle_command_reply: tid=0 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 client.16032 shutdown 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 msgr2=0x55d6cd0cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0cefb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f77580e2270 tx=0x7f7758010000 comp rx=0 tx=0).stop 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 shutdown_connections 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd0df2b0 0x55d6cd0e16a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/1077284587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0cefb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 >> 172.21.15.99:0/1077284587 conn(0x55d6ccff2f40 msgr2=0x55d6cd0bb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f774f7fe700 0 client.16032 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:04.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 shutdown_connections 2024-05-14T17:10:04.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 1 -- 172.21.15.99:0/1077284587 wait complete. 2024-05-14T17:10:04.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.748+0000 7f7780cd5b80 20 client.16032 trim_cache size 0 max 16384 2024-05-14T17:10:04.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 Processor -- start 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 -- start start 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0bb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b28b0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77701b0110 con 0x55d6cd0cadd0 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.768+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77701adf60 con 0x55d6cd0c6350 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43126/0 (socket says 172.21.15.99:43126) 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 -- 172.21.15.99:0/2699741704 learned_addr learned my addr 172.21.15.99:0/2699741704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f776ffff700 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0bb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f77749dc700 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 -- 172.21.15.99:0/2699741704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 msgr2=0x55d6cd0bb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0bb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 -- 172.21.15.99:0/2699741704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0cadd0 msgr2=0x55d6cd0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 -- 172.21.15.99:0/2699741704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6cd0bb6a0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7767fff700 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0c3d20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f77580a8f10 tx=0x7f77580104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea767707652b014e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7765ffb700 1 -- 172.21.15.99:0/2699741704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580c2070 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7765ffb700 1 -- 172.21.15.99:0/2699741704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77580b1e40 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.769+0000 7f7765ffb700 1 -- 172.21.15.99:0/2699741704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758020070 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 -- 172.21.15.99:0/2699741704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 msgr2=0x55d6cd0c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0c3d20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f77580a8f10 tx=0x7f77580104e0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 -- 172.21.15.99:0/2699741704 shutdown_connections 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0c6350 0x55d6cd0bb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/2699741704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0c3d20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 -- 172.21.15.99:0/2699741704 >> 172.21.15.99:0/2699741704 conn(0x55d6ccff2f40 msgr2=0x55d6cd0daae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 -- 172.21.15.99:0/2699741704 shutdown_connections 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 -- 172.21.15.99:0/2699741704 wait complete. 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.770+0000 7f7780cd5b80 1 -- start start 2024-05-14T17:10:04.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 Processor -- start 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77701ae5d0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77701afa40 con 0x55d6cd0c6350 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77701aeca0 con 0x55d6cd0cadd0 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7780cd5b80 10 client.0 _ll_register_callbacks cb 0x55d6ccfaeae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f776ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43140/0 (socket says 172.21.15.99:43140) 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 -- 172.21.15.99:0/3828167661 learned_addr learned my addr 172.21.15.99:0/3828167661 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f77749dc700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 -- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 msgr2=0x55d6cd0e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 -- 172.21.15.99:0/3828167661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 msgr2=0x55d6cd0e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 -- 172.21.15.99:0/3828167661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6cd0e1290 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f776ffff700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f7767fff700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0e49e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7758012040 tx=0x7f7758010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f8953d963bf558 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.771+0000 7f774f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f774f7fe700 1 -- 172.21.15.99:0/3828167661 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580c2070 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f774f7fe700 1 -- 172.21.15.99:0/3828167661 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8477+0+0 (secure 0 0 0) 0x7f77580a84c0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77580b1e40 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f774f7fe700 1 -- 172.21.15.99:0/3828167661 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7758020070 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 10 client.16038 fetch_fsmap learned FSMap version 28 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6cd0e1290 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f774f7fe700 1 -- 172.21.15.99:0/3828167661 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77580b1e40 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f774f7fe700 1 -- 172.21.15.99:0/3828167661 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f775800f9b0 con 0x55d6cd0ca4b0 2024-05-14T17:10:04.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 10 client.16038 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 20 client.16038 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd14a7f0 0x55d6cd14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 4 client.16038 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.7, "await_for": 45.0}] 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.7, "await_for": 45.0}) -- 0x55d6cd01f630 con 0x55d6cd14a7f0 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.772+0000 7f776ffff700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd14a7f0 0x55d6cd14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.773+0000 7f774f7fe700 10 client.16038 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:04.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.773+0000 7f776ffff700 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd14a7f0 0x55d6cd14cc10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f7d6a8807e0982 server_cookie=ba8fda1c8a346e4b in_seq=0 out_seq=0 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f774f7fe700 1 -- 172.21.15.99:0/3828167661 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d6cd01f630 con 0x55d6cd14a7f0 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f774f7fe700 10 client.16038 handle_command_reply: tid=0 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 client.16038 shutdown 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 msgr2=0x55d6cd0e49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0e49e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7758012040 tx=0x7f7758010000 comp rx=0 tx=0).stop 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 shutdown_connections 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6cd0cadd0 0x55d6cd0e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d6cd0c6350 0x55d6cd0e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55d6cd14a7f0 0x55d6cd14cc10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 --2- 172.21.15.99:0/3828167661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d6cd0ca4b0 0x55d6cd0e49e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 >> 172.21.15.99:0/3828167661 conn(0x55d6ccff2f40 msgr2=0x55d6cd0db370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f774f7fe700 0 client.16038 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 shutdown_connections 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 1 -- 172.21.15.99:0/3828167661 wait complete. 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.837+0000 7f7780cd5b80 20 client.16038 trim_cache size 0 max 16384 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 29, 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 18, 2024-05-14T17:10:04.838 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "cbc14f15": { 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 18, 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 133.7, 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:10:04.839 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f775c0b2b20 msgr2=0x7f775c0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f775c0b2b20 0x7f775c0b4fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7760011490 tx=0x7f7760009260 comp rx=0 tx=0).stop 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 msgr2=0x7f777019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 0x7f777019c310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f775800d980 tx=0x7f775800dc90 comp rx=0 tx=0).stop 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 shutdown_connections 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f775c0b2b20 0x7f775c0b4fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7770108840 0x7f77701a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7770103850 0x7f777019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 --2- 172.21.15.99:0/3194906495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7770104200 0x7f777019c310 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 >> 172.21.15.99:0/3194906495 conn(0x7f777007a610 msgr2=0x7f7770101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 shutdown_connections 2024-05-14T17:10:04.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:04.838+0000 7f777643f700 1 -- 172.21.15.99:0/3194906495 wait complete. 2024-05-14T17:10:05.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:04 smithi099 ceph-mon[31851]: pgmap v196: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-05-14T17:10:05.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:04 smithi150 ceph-mon[36204]: pgmap v196: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-05-14T17:10:05.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:04 smithi193 ceph-mon[35962]: pgmap v196: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-05-14T17:10:05.482 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cbc14f15', quiesce took 5.8 seconds. Will release after: 7.8999999999999995 2024-05-14T17:10:05.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.72 MB/sec execute 137 sec latency 3036.910 ms 2024-05-14T17:10:06.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:05 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:06.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:05 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:06.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:05 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:06.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.66 MB/sec execute 138 sec latency 1668.111 ms 2024-05-14T17:10:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:06 smithi150 ceph-mon[36204]: pgmap v197: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-05-14T17:10:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:06 smithi150 ceph-mon[36204]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T17:10:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:06 smithi099 ceph-mon[31851]: pgmap v197: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-05-14T17:10:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:06 smithi099 ceph-mon[31851]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T17:10:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:06 smithi193 ceph-mon[35962]: pgmap v197: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-05-14T17:10:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:06 smithi193 ceph-mon[35962]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T17:10:07.531 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.60 MB/sec execute 139 sec latency 2668.190 ms 2024-05-14T17:10:08.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.55 MB/sec execute 140 sec latency 3668.272 ms 2024-05-14T17:10:09.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:08 smithi150 ceph-mon[36204]: pgmap v199: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-05-14T17:10:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:08 smithi099 ceph-mon[31851]: pgmap v199: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-05-14T17:10:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:08 smithi193 ceph-mon[35962]: pgmap v199: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-05-14T17:10:09.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.50 MB/sec execute 141 sec latency 4668.373 ms 2024-05-14T17:10:10.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.44 MB/sec execute 142 sec latency 5668.484 ms 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 -- 172.21.15.99:0/3010412525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 msgr2=0x7fdb001040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/3010412525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb001040c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009900 tx=0x7fdaf002a480 comp rx=0 tx=0).stop 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 -- 172.21.15.99:0/3010412525 shutdown_connections 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/3010412525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 0x7fdb0010f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/3010412525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb00104600 0x7fdb0010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/3010412525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb001040c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 -- 172.21.15.99:0/3010412525 >> 172.21.15.99:0/3010412525 conn(0x7fdb00078b60 msgr2=0x7fdb00078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 -- 172.21.15.99:0/3010412525 shutdown_connections 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 -- 172.21.15.99:0/3010412525 wait complete. 2024-05-14T17:10:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.914+0000 7fdb05bdd640 1 Processor -- start 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 -- start start 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb00199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb00104600 0x7fdb001996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 0x7fdb0019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00112cf0 con 0x7fdb00103ce0 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00112b70 con 0x7fdb0010d0b0 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdb05bdd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00112e70 con 0x7fdb00104600 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb00199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb00199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43158/0 (socket says 172.21.15.99:43158) 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 -- 172.21.15.99:0/674554704 learned_addr learned my addr 172.21.15.99:0/674554704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdafeffd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb00104600 0x7fdb001996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdafffff640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 0x7fdb0019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 -- 172.21.15.99:0/674554704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb00104600 msgr2=0x7fdb001996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb00104600 0x7fdb001996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 -- 172.21.15.99:0/674554704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 msgr2=0x7fdb0019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 0x7fdb0019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdaff7fe640 1 -- 172.21.15.99:0/674554704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0019e180 con 0x7fdb00103ce0 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.915+0000 7fdafffff640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 0x7fdb0019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.916+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb00199190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005940 tx=0x7fdaf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.916+0000 7fdafcff9640 1 -- 172.21.15.99:0/674554704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf003f030 con 0x7fdb00103ce0 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.916+0000 7fdafcff9640 1 -- 172.21.15.99:0/674554704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf0003cb0 con 0x7fdb00103ce0 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.916+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0019e410 con 0x7fdb00103ce0 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.916+0000 7fdafcff9640 1 -- 172.21.15.99:0/674554704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0035070 con 0x7fdb00103ce0 2024-05-14T17:10:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.916+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001a5d00 con 0x7fdb00103ce0 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.918+0000 7fdafcff9640 1 -- 172.21.15.99:0/674554704 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdaf0042030 con 0x7fdb00103ce0 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.918+0000 7fdb089d4740 1 Processor -- start 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.918+0000 7fdb089d4740 1 -- start start 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea496c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 0x55adea4a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdb089d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55adea409c30 con 0x55adea497150 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdb089d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55adea3f97d0 con 0x55adea496830 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdb089d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55adea3fd810 con 0x55adea4a5ee0 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 0x55adea4a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 0x55adea4a59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43164/0 (socket says 172.21.15.99:43164) 2024-05-14T17:10:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 -- 172.21.15.99:0/2876916624 learned_addr learned my addr 172.21.15.99:0/2876916624 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdafeffd640 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdafffff640 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea496c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdafcff9640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdaec0b2a90 0x7fdaec0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 -- 172.21.15.99:0/2876916624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 msgr2=0x55adea4a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 -- 172.21.15.99:0/2876916624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 msgr2=0x55adea496c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea496c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 -- 172.21.15.99:0/2876916624 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adea10e210 con 0x55adea497150 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdafffff640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdaec0b2a90 0x7fdaec0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdafcff9640 1 -- 172.21.15.99:0/674554704 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fdaf0102e30 con 0x7fdb00103ce0 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.919+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 0x55adea4a59a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdaf01004f0 tx=0x7fdaf008f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b589a18e67776618 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdaf5ffb640 1 -- 172.21.15.99:0/2876916624 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00c9a60 con 0x55adea497150 2024-05-14T17:10:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdaf5ffb640 1 -- 172.21.15.99:0/2876916624 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf00c9c20 con 0x55adea497150 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdaf5ffb640 1 -- 172.21.15.99:0/2876916624 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005d50 con 0x55adea497150 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 -- 172.21.15.99:0/2876916624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 msgr2=0x55adea4a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdafffff640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdaec0b2a90 0x7fdaec0b4f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009370 tx=0x7fdaf8004a40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 0x55adea4a59a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdaf01004f0 tx=0x7fdaf008f6b0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 -- 172.21.15.99:0/2876916624 shutdown_connections 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea497150 0x55adea4a59a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2876916624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea496c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 -- 172.21.15.99:0/2876916624 >> 172.21.15.99:0/2876916624 conn(0x55adea492e00 msgr2=0x55adea4931e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 -- 172.21.15.99:0/2876916624 shutdown_connections 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.920+0000 7fdb089d4740 1 -- 172.21.15.99:0/2876916624 wait complete. 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 -- start start 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 Processor -- start 2024-05-14T17:10:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55adea0df790 con 0x55adea496830 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55adea3b5890 con 0x55adea497150 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55adea4a8870 con 0x55adea4a5ee0 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdb089d4740 10 client.0 _ll_register_callbacks cb 0x55adea3b8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43172/0 (socket says 172.21.15.99:43172) 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.921+0000 7fdaff7fe640 1 -- 172.21.15.99:0/106760100 learned_addr learned my addr 172.21.15.99:0/106760100 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdafeffd640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdafffff640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdaff7fe640 1 -- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 msgr2=0x55adea4a2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdaff7fe640 1 -- 172.21.15.99:0/106760100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 msgr2=0x55adea4a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdaff7fe640 1 -- 172.21.15.99:0/106760100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adea1da620 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a57b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0032af0 tx=0x7fdaf00c9fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4bcf884d1d9a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdadeffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdadeffd640 1 -- 172.21.15.99:0/106760100 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00c9880 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdadeffd640 1 -- 172.21.15.99:0/106760100 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fdaf00c63a0 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf00c9c20 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdadeffd640 1 -- 172.21.15.99:0/106760100 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdaf0005d50 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdb089d4740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adea4b4290 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdadeffd640 1 -- 172.21.15.99:0/106760100 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf00c9c20 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.922+0000 7fdadeffd640 1 -- 172.21.15.99:0/106760100 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdaf003bbf0 con 0x55adea496830 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdb089d4740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdb089d4740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdb089d4740 20 client.16050 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdb089d4740 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea4b55e0 0x55adea4b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdb089d4740 4 client.16050 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adea40bb80 con 0x55adea4b55e0 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdafeffd640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea4b55e0 0x55adea4b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdadeffd640 10 client.16050 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.923+0000 7fdafeffd640 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea4b55e0 0x55adea4b79f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4485df7c53e601 server_cookie=f4b0d4b5b7fa6bbd in_seq=0 out_seq=0 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdadeffd640 1 -- 172.21.15.99:0/106760100 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55adea40bb80 con 0x55adea4b55e0 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdadeffd640 10 client.16050 handle_command_reply: tid=0 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 client.16050 shutdown 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 msgr2=0x55adea4a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a57b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0032af0 tx=0x7fdaf00c9fe0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 shutdown_connections 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea4b55e0 0x55adea4b79f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdadeffd640 0 client.16050 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 --2- 172.21.15.99:0/106760100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a57b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.924+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 >> 172.21.15.99:0/106760100 conn(0x55adea492e00 msgr2=0x55adea492100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.925+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 shutdown_connections 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.925+0000 7fdb089d4740 1 -- 172.21.15.99:0/106760100 wait complete. 2024-05-14T17:10:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.925+0000 7fdb089d4740 20 client.16050 trim_cache size 0 max 16384 2024-05-14T17:10:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 Processor -- start 2024-05-14T17:10:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 -- start start 2024-05-14T17:10:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea49e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea497150 0x55adea4a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 0x55adea498990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001aed60 con 0x55adea4a5ee0 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001ac5d0 con 0x55adea496830 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdb089d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001aa440 con 0x55adea497150 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 0x55adea498990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 0x55adea498990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43182/0 (socket says 172.21.15.99:43182) 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 -- 172.21.15.99:0/2834549898 learned_addr learned my addr 172.21.15.99:0/2834549898 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 -- 172.21.15.99:0/2834549898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea497150 msgr2=0x55adea4a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea497150 0x55adea4a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 -- 172.21.15.99:0/2834549898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 msgr2=0x55adea49e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea49e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 -- 172.21.15.99:0/2834549898 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adea48e830 con 0x55adea4a5ee0 2024-05-14T17:10:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdafffff640 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 0x55adea498990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdaf801e510 tx=0x7fdaf8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e3fbc42bbdbe49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.940+0000 7fdaf57fa640 1 -- 172.21.15.99:0/2834549898 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf801b660 con 0x55adea4a5ee0 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdaf57fa640 1 -- 172.21.15.99:0/2834549898 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf801bcb0 con 0x55adea4a5ee0 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdaf57fa640 1 -- 172.21.15.99:0/2834549898 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8012dc0 con 0x55adea4a5ee0 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 -- 172.21.15.99:0/2834549898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 msgr2=0x55adea498990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 0x55adea498990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdaf801e510 tx=0x7fdaf8013000 comp rx=0 tx=0).stop 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 -- 172.21.15.99:0/2834549898 shutdown_connections 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea4a5ee0 0x55adea498990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea497150 0x55adea4a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 --2- 172.21.15.99:0/2834549898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea496830 0x55adea49e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 -- 172.21.15.99:0/2834549898 >> 172.21.15.99:0/2834549898 conn(0x55adea492e00 msgr2=0x55adea498280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 -- 172.21.15.99:0/2834549898 shutdown_connections 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 -- 172.21.15.99:0/2834549898 wait complete. 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.941+0000 7fdb089d4740 1 -- start start 2024-05-14T17:10:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 Processor -- start 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4aa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001aaa90 con 0x55adea496830 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001abf00 con 0x55adea497150 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001ab160 con 0x55adea4a5ee0 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdb089d4740 10 client.0 _ll_register_callbacks cb 0x7fdaf00c6060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43184/0 (socket says 172.21.15.99:43184) 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 -- 172.21.15.99:0/4056434919 learned_addr learned my addr 172.21.15.99:0/4056434919 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdafeffd640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 -- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 msgr2=0x55adea4aa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 -- 172.21.15.99:0/4056434919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 msgr2=0x55adea4a9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 -- 172.21.15.99:0/4056434919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adea51eeb0 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdafeffd640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdaff7fe640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a95c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0006010 tx=0x7fdaf00c65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facda7d43c581453 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdadeffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.942+0000 7fdadeffd640 1 -- 172.21.15.99:0/4056434919 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00c9a60 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdadeffd640 1 -- 172.21.15.99:0/4056434919 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fdaf003bbf0 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf801bcb0 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdadeffd640 1 -- 172.21.15.99:0/4056434919 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdaf00c9880 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 10 client.16056 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adea51eeb0 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdadeffd640 1 -- 172.21.15.99:0/4056434919 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf801bcb0 con 0x55adea496830 2024-05-14T17:10:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdadeffd640 1 -- 172.21.15.99:0/4056434919 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdaf00ca370 con 0x55adea496830 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 20 client.16056 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea520490 0x55adea5228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 4 client.16056 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adea405360 con 0x55adea520490 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.943+0000 7fdafeffd640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea520490 0x55adea5228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.944+0000 7fdadeffd640 10 client.16056 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.944+0000 7fdafeffd640 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea520490 0x55adea5228b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55bbe23ded429a0 server_cookie=53be8bb73694a60b in_seq=0 out_seq=0 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdadeffd640 1 -- 172.21.15.99:0/4056434919 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55adea405360 con 0x55adea520490 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdadeffd640 10 client.16056 handle_command_reply: tid=0 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 client.16056 shutdown 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 msgr2=0x55adea4a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a95c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0006010 tx=0x7fdaf00c65d0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 shutdown_connections 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55adea520490 0x55adea5228b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adea4a5ee0 0x55adea4aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdadeffd640 0 client.16056 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55adea497150 0x55adea4a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 --2- 172.21.15.99:0/4056434919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adea496830 0x55adea4a95c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 >> 172.21.15.99:0/4056434919 conn(0x55adea492e00 msgr2=0x55adea498a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 shutdown_connections 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 1 -- 172.21.15.99:0/4056434919 wait complete. 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.974+0000 7fdb089d4740 20 client.16056 trim_cache size 0 max 16384 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:10:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:10:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdaec0b2a90 msgr2=0x7fdaec0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdaec0b2a90 0x7fdaec0b4f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009370 tx=0x7fdaf8004a40 comp rx=0 tx=0).stop 2024-05-14T17:10:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 msgr2=0x7fdb00199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb00199190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005940 tx=0x7fdaf00042f0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 shutdown_connections 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdaec0b2a90 0x7fdaec0b4f50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0010d0b0 0x7fdb0019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb00104600 0x7fdb001996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 --2- 172.21.15.99:0/674554704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb00103ce0 0x7fdb00199190 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 >> 172.21.15.99:0/674554704 conn(0x7fdb00078b60 msgr2=0x7fdb000fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 shutdown_connections 2024-05-14T17:10:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:10.975+0000 7fdb05bdd640 1 -- 172.21.15.99:0/674554704 wait complete. 2024-05-14T17:10:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:10 smithi150 ceph-mon[36204]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-05-14T17:10:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:10 smithi150 ceph-mon[36204]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T17:10:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:10 smithi099 ceph-mon[31851]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-05-14T17:10:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:10 smithi099 ceph-mon[31851]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T17:10:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:10 smithi193 ceph-mon[35962]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2024-05-14T17:10:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:10 smithi193 ceph-mon[35962]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T17:10:11.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.39 MB/sec execute 143 sec latency 6668.559 ms 2024-05-14T17:10:12.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.34 MB/sec execute 144 sec latency 7668.657 ms 2024-05-14T17:10:13.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:12 smithi150 ceph-mon[36204]: pgmap v202: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-05-14T17:10:13.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:12 smithi099 ceph-mon[31851]: pgmap v202: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-05-14T17:10:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:12 smithi193 ceph-mon[35962]: pgmap v202: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-05-14T17:10:13.383 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:10:13.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.29 MB/sec execute 145 sec latency 8668.702 ms 2024-05-14T17:10:13.633 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:10:14.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.24 MB/sec execute 146 sec latency 9668.771 ms 2024-05-14T17:10:14.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:14 smithi099 ceph-mon[31851]: pgmap v203: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 -- 172.21.15.99:0/1981247759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2801026a0 msgr2=0x7fb280108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 --2- 172.21.15.99:0/1981247759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2801026a0 0x7fb280108200 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb2700099c0 tx=0x7fb270009cd0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 -- 172.21.15.99:0/1981247759 shutdown_connections 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 --2- 172.21.15.99:0/1981247759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb280108930 0x7fb2801104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 --2- 172.21.15.99:0/1981247759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb280101cf0 0x7fb2801020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 --2- 172.21.15.99:0/1981247759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2801026a0 0x7fb280108200 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 -- 172.21.15.99:0/1981247759 >> 172.21.15.99:0/1981247759 conn(0x7fb280078a40 msgr2=0x7fb2800ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 -- 172.21.15.99:0/1981247759 shutdown_connections 2024-05-14T17:10:15.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 -- 172.21.15.99:0/1981247759 wait complete. 2024-05-14T17:10:15.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.199+0000 7fb285e1a700 1 Processor -- start 2024-05-14T17:10:15.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 -- start start 2024-05-14T17:10:15.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 0x7fb28019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2801026a0 0x7fb28019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb280108930 0x7fb2801a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2801136f0 con 0x7fb280101cf0 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb280113570 con 0x7fb280108930 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb285e1a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb280113870 con 0x7fb2801026a0 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 0x7fb28019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 0x7fb28019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35858/0 (socket says 172.21.15.99:35858) 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 -- 172.21.15.99:0/2191650035 learned_addr learned my addr 172.21.15.99:0/2191650035 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27effd700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2801026a0 0x7fb28019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27ffff700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb280108930 0x7fb2801a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 -- 172.21.15.99:0/2191650035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2801026a0 msgr2=0x7fb28019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2801026a0 0x7fb28019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 -- 172.21.15.99:0/2191650035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb280108930 msgr2=0x7fb2801a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb280108930 0x7fb2801a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.200+0000 7fb27f7fe700 1 -- 172.21.15.99:0/2191650035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2801a0db0 con 0x7fb280101cf0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.201+0000 7fb27f7fe700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 0x7fb28019bda0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb27400d950 tx=0x7fb27400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.201+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274016070 con 0x7fb280101cf0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.201+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb274009df0 con 0x7fb280101cf0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.201+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27400f3c0 con 0x7fb280101cf0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.201+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2801a10a0 con 0x7fb280101cf0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.201+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2801a8920 con 0x7fb280101cf0 2024-05-14T17:10:15.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.202+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28006a230 con 0x7fb280101cf0 2024-05-14T17:10:15.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:14 smithi150 ceph-mon[36204]: pgmap v203: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-05-14T17:10:15.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.203+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb274019070 con 0x7fb280101cf0 2024-05-14T17:10:15.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.204+0000 7fb27cff9700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2680b29f0 0x7fb2680b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.204+0000 7fb27effd700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2680b29f0 0x7fb2680b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.204+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb2740e2270 con 0x7fb280101cf0 2024-05-14T17:10:15.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.205+0000 7fb27effd700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2680b29f0 0x7fb2680b4eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb270009990 tx=0x7fb270005c40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:15.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.208+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2740a9bf0 con 0x7fb280101cf0 2024-05-14T17:10:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:14 smithi193 ceph-mon[35962]: pgmap v203: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-05-14T17:10:15.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.475+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2801a8af0 con 0x7fb280101cf0 2024-05-14T17:10:15.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.476+0000 7fb27cff9700 1 -- 172.21.15.99:0/2191650035 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11879 (secure 0 0 0) 0x7fb2740a94f0 con 0x7fb280101cf0 2024-05-14T17:10:15.477 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:10:15.477 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:10:04.586724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:10:15.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.478+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2680b29f0 msgr2=0x7fb2680b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.478+0000 7fb285e1a700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2680b29f0 0x7fb2680b4eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb270009990 tx=0x7fb270005c40 comp rx=0 tx=0).stop 2024-05-14T17:10:15.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.478+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 msgr2=0x7fb28019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.478+0000 7fb285e1a700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 0x7fb28019bda0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb27400d950 tx=0x7fb27400dc60 comp rx=0 tx=0).stop 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 shutdown_connections 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb2680b29f0 0x7fb2680b4eb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb280108930 0x7fb2801a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280101cf0 0x7fb28019bda0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 --2- 172.21.15.99:0/2191650035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2801026a0 0x7fb28019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 >> 172.21.15.99:0/2191650035 conn(0x7fb280078a40 msgr2=0x7fb2801070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 shutdown_connections 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:15.479+0000 7fb285e1a700 1 -- 172.21.15.99:0/2191650035 wait complete. 2024-05-14T17:10:15.480 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-05-14T17:10:15.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.19 MB/sec execute 147 sec latency 10668.809 ms 2024-05-14T17:10:16.036 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:15 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2191650035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:16.038 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id cbc14f15 --release --await' 2024-05-14T17:10:16.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id cbc14f15 --release --await 2024-05-14T17:10:16.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:15 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2191650035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:10:16.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:15 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2191650035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:16.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.14 MB/sec execute 148 sec latency 11668.876 ms 2024-05-14T17:10:17.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:16 smithi099 ceph-mon[31851]: pgmap v204: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-05-14T17:10:17.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:16 smithi150 ceph-mon[36204]: pgmap v204: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-05-14T17:10:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:16 smithi193 ceph-mon[35962]: pgmap v204: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-05-14T17:10:17.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 335677 7.09 MB/sec execute 149 sec latency 12668.962 ms 2024-05-14T17:10:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 -- 172.21.15.99:0/3667086933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102440 msgr2=0x7f35e81028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 --2- 172.21.15.99:0/3667086933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102440 0x7f35e81028c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009990 tx=0x7f35d8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 -- 172.21.15.99:0/3667086933 shutdown_connections 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 --2- 172.21.15.99:0/3667086933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102e00 0x7f35e810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 --2- 172.21.15.99:0/3667086933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e8108520 0x7f35e8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 --2- 172.21.15.99:0/3667086933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102440 0x7f35e81028c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 -- 172.21.15.99:0/3667086933 >> 172.21.15.99:0/3667086933 conn(0x7f35e80793f0 msgr2=0x7f35e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 -- 172.21.15.99:0/3667086933 shutdown_connections 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 -- 172.21.15.99:0/3667086933 wait complete. 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.944+0000 7f35efd6f700 1 Processor -- start 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 -- start start 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102440 0x7f35e819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 0x7f35e819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e8108520 0x7f35e81a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81125f0 con 0x7f35e8102e00 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8112470 con 0x7f35e8108520 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35efd6f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8112770 con 0x7f35e8102440 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 0x7f35e819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 0x7f35e819af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35874/0 (socket says 172.21.15.99:35874) 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 -- 172.21.15.99:0/2561049049 learned_addr learned my addr 172.21.15.99:0/2561049049 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35edb0b700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102440 0x7f35e819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ee30c700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e8108520 0x7f35e81a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 -- 172.21.15.99:0/2561049049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102440 msgr2=0x7f35e819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102440 0x7f35e819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 -- 172.21.15.99:0/2561049049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e8108520 msgr2=0x7f35e81a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e8108520 0x7f35e81a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 -- 172.21.15.99:0/2561049049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e81a5ff0 con 0x7f35e8102e00 2024-05-14T17:10:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.945+0000 7f35ed30a700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 0x7f35e819af70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.946+0000 7f35deffd700 1 -- 172.21.15.99:0/2561049049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8027070 con 0x7f35e8102e00 2024-05-14T17:10:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.946+0000 7f35deffd700 1 -- 172.21.15.99:0/2561049049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d8003fe0 con 0x7f35e8102e00 2024-05-14T17:10:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.946+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e81a61c0 con 0x7f35e8102e00 2024-05-14T17:10:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.946+0000 7f35deffd700 1 -- 172.21.15.99:0/2561049049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8022e30 con 0x7f35e8102e00 2024-05-14T17:10:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.946+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a6610 con 0x7f35e8102e00 2024-05-14T17:10:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.948+0000 7f35deffd700 1 -- 172.21.15.99:0/2561049049 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f35d802a070 con 0x7f35e8102e00 2024-05-14T17:10:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.949+0000 7f35deffd700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f35d40b29f0 0x7f35d40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.949+0000 7f35edb0b700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f35d40b29f0 0x7f35d40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.949+0000 7f35deffd700 1 -- 172.21.15.99:0/2561049049 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f35d80e3360 con 0x7f35e8102e00 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.949+0000 7f35edb0b700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f35d40b29f0 0x7f35d40b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35e400abd0 tx=0x7f35e400a380 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.949+0000 7f35fa605b80 1 Processor -- start 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.949+0000 7f35fa605b80 1 -- start start 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc861800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc85d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35fa605b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5638fc7bae70 con 0x5638fc85d2c0 2024-05-14T17:10:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35fa605b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5638fc7aaa10 con 0x5638fc861420 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35fa605b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638fc7aea50 con 0x5638fc861d40 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc85d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc85d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35886/0 (socket says 172.21.15.99:35886) 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 -- 172.21.15.99:0/1206282487 learned_addr learned my addr 172.21.15.99:0/1206282487 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ed30a700 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35edb0b700 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc861800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 -- 172.21.15.99:0/1206282487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 msgr2=0x5638fc85cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 -- 172.21.15.99:0/1206282487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 msgr2=0x5638fc861800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc861800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 -- 172.21.15.99:0/1206282487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fc7540b0 con 0x5638fc85d2c0 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35edb0b700 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc861800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35ee30c700 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc85d720 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f35e000a730 tx=0x7f35e00078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a707c31143bd73 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.950+0000 7f35d3fff700 1 -- 172.21.15.99:0/1206282487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0003a00 con 0x5638fc85d2c0 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35d3fff700 1 -- 172.21.15.99:0/1206282487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e0003ba0 con 0x5638fc85d2c0 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35d3fff700 1 -- 172.21.15.99:0/1206282487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0014660 con 0x5638fc85d2c0 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 -- 172.21.15.99:0/1206282487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 msgr2=0x5638fc85d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc85d720 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f35e000a730 tx=0x7f35e00078d0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 -- 172.21.15.99:0/1206282487 shutdown_connections 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc85d720 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc861800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1206282487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 -- 172.21.15.99:0/1206282487 >> 172.21.15.99:0/1206282487 conn(0x5638fc789eb0 msgr2=0x5638fc8574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 -- 172.21.15.99:0/1206282487 shutdown_connections 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.951+0000 7f35fa605b80 1 -- 172.21.15.99:0/1206282487 wait complete. 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 -- start start 2024-05-14T17:10:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 Processor -- start 2024-05-14T17:10:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc868ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861420 0x5638fc865e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861d40 0x5638fc8663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5638fc6b10a0 con 0x5638fc85d2c0 2024-05-14T17:10:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5638fc712ea0 con 0x5638fc861d40 2024-05-14T17:10:17.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638fc869820 con 0x5638fc861420 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35fa605b80 10 client.0 _ll_register_callbacks cb 0x5638fc745a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35edb0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc868ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35edb0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc868ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35898/0 (socket says 172.21.15.99:35898) 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35edb0b700 1 -- 172.21.15.99:0/154911862 learned_addr learned my addr 172.21.15.99:0/154911862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35ed30a700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861420 0x5638fc865e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.952+0000 7f35ee30c700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861d40 0x5638fc8663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35edb0b700 1 -- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861420 msgr2=0x5638fc865e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35edb0b700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861420 0x5638fc865e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35edb0b700 1 -- 172.21.15.99:0/154911862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861d40 msgr2=0x5638fc8663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35edb0b700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861d40 0x5638fc8663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35edb0b700 1 -- 172.21.15.99:0/154911862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fc6fa9d0 con 0x5638fc85d2c0 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35ed30a700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861420 0x5638fc865e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:17.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35edb0b700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc868ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f35e400ffa0 tx=0x7f35e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819b4e0f6fd88dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35d17fa700 1 -- 172.21.15.99:0/154911862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4016070 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35d17fa700 1 -- 172.21.15.99:0/154911862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f35e4010840 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e0003ba0 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35d17fa700 1 -- 172.21.15.99:0/154911862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35e4010a80 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35fa605b80 10 client.16074 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638fc764860 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.953+0000 7f35d17fa700 1 -- 172.21.15.99:0/154911862 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e0003ba0 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35d17fa700 1 -- 172.21.15.99:0/154911862 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f35e4019030 con 0x5638fc85d2c0 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35fa605b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35fa605b80 10 client.16074 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35fa605b80 20 client.16074 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:17.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35fa605b80 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8762c0 0x5638fc878730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35fa605b80 4 client.16074 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638fc7bcdc0 con 0x5638fc8762c0 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35ed30a700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8762c0 0x5638fc878730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35d17fa700 10 client.16074 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.954+0000 7f35ed30a700 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8762c0 0x5638fc878730 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a131e23d9f3390d0 server_cookie=f346a68379a53830 in_seq=0 out_seq=0 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35d17fa700 1 -- 172.21.15.99:0/154911862 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638fc7bcdc0 con 0x5638fc8762c0 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35d17fa700 10 client.16074 handle_command_reply: tid=0 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 client.16074 shutdown 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 msgr2=0x5638fc868ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc868ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f35e400ffa0 tx=0x7f35e40047c0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 shutdown_connections 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861d40 0x5638fc8663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8762c0 0x5638fc878730 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc868ac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 --2- 172.21.15.99:0/154911862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861420 0x5638fc865e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35d17fa700 0 client.16074 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 >> 172.21.15.99:0/154911862 conn(0x5638fc789eb0 msgr2=0x5638fc851fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 shutdown_connections 2024-05-14T17:10:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.955+0000 7f35fa605b80 1 -- 172.21.15.99:0/154911862 wait complete. 2024-05-14T17:10:17.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.956+0000 7f35fa605b80 20 client.16074 trim_cache size 0 max 16384 2024-05-14T17:10:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.975+0000 7f35fa605b80 1 Processor -- start 2024-05-14T17:10:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.975+0000 7f35fa605b80 1 -- start start 2024-05-14T17:10:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc872730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc8520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35fa605b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81b0610 con 0x5638fc85d2c0 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35fa605b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ade40 con 0x5638fc861420 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35fa605b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81abcb0 con 0x5638fc861d40 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc872730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc872730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35900/0 (socket says 172.21.15.99:35900) 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 -- 172.21.15.99:0/1380267642 learned_addr learned my addr 172.21.15.99:0/1380267642 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35ee30c700 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35ed30a700 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc8520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 -- 172.21.15.99:0/1380267642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 msgr2=0x5638fc85ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 -- 172.21.15.99:0/1380267642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 msgr2=0x5638fc8520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc8520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 -- 172.21.15.99:0/1380267642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fc873d10 con 0x5638fc85d2c0 2024-05-14T17:10:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35ee30c700 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35edb0b700 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc872730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f35e400ff70 tx=0x7f35e400a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6d3b79bb046a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35d37fe700 1 -- 172.21.15.99:0/1380267642 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4016040 con 0x5638fc85d2c0 2024-05-14T17:10:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35d37fe700 1 -- 172.21.15.99:0/1380267642 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e4021070 con 0x5638fc85d2c0 2024-05-14T17:10:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.976+0000 7f35d37fe700 1 -- 172.21.15.99:0/1380267642 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40267b0 con 0x5638fc85d2c0 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 -- 172.21.15.99:0/1380267642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 msgr2=0x5638fc872730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc872730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f35e400ff70 tx=0x7f35e400a820 comp rx=0 tx=0).stop 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 -- 172.21.15.99:0/1380267642 shutdown_connections 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc85ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc85d2c0 0x5638fc872730 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 --2- 172.21.15.99:0/1380267642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc861420 0x5638fc8520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 -- 172.21.15.99:0/1380267642 >> 172.21.15.99:0/1380267642 conn(0x5638fc789eb0 msgr2=0x5638fc860970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 -- 172.21.15.99:0/1380267642 shutdown_connections 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.977+0000 7f35fa605b80 1 -- 172.21.15.99:0/1380267642 wait complete. 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 -- start start 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 Processor -- start 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc85d2c0 0x5638fc8dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 0x5638fc8db100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc8db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ac300 con 0x5638fc861420 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ad770 con 0x5638fc85d2c0 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35ed30a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 0x5638fc8db100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.978+0000 7f35fa605b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ac9d0 con 0x5638fc861d40 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 0x5638fc8db100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35910/0 (socket says 172.21.15.99:35910) 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 -- 172.21.15.99:0/79736580 learned_addr learned my addr 172.21.15.99:0/79736580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35fa605b80 10 client.0 _ll_register_callbacks cb 0x5638fc745a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35edb0b700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc85d2c0 0x5638fc8dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 -- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 msgr2=0x5638fc8db640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ee30c700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc8db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc8db640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 -- 172.21.15.99:0/79736580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc85d2c0 msgr2=0x5638fc8dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc85d2c0 0x5638fc8dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 -- 172.21.15.99:0/79736580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fc86daa0 con 0x5638fc861420 2024-05-14T17:10:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35edb0b700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc85d2c0 0x5638fc8dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35ed30a700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 0x5638fc8db100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f35d80aa3d0 tx=0x7f35d80b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44791583c103cf58 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35d17fa700 1 -- 172.21.15.99:0/79736580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80aea80 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35d17fa700 1 -- 172.21.15.99:0/79736580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f35d8022470 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e4021070 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35d17fa700 1 -- 172.21.15.99:0/79736580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35d80b34a0 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35fa605b80 10 client.16080 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.979+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638fc86ed90 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35d17fa700 1 -- 172.21.15.99:0/79736580 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e4021070 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35d17fa700 1 -- 172.21.15.99:0/79736580 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f35d80aa930 con 0x5638fc861420 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35fa605b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35fa605b80 10 client.16080 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35fa605b80 20 client.16080 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35fa605b80 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8e7010 0x5638fc8e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35fa605b80 4 client.16080 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cbc14f15", "release": true, "await": true}] 2024-05-14T17:10:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cbc14f15", "release": true, "await": true}) -- 0x5638fc7b65a0 con 0x5638fc8e7010 2024-05-14T17:10:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35edb0b700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8e7010 0x5638fc8e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35d17fa700 10 client.16080 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.980+0000 7f35edb0b700 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8e7010 0x5638fc8e9400 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd7596292fe425a server_cookie=d2884a88198c8bc1 in_seq=0 out_seq=0 2024-05-14T17:10:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35d17fa700 1 -- 172.21.15.99:0/79736580 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5638fc7b65a0 con 0x5638fc8e7010 2024-05-14T17:10:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35d17fa700 10 client.16080 handle_command_reply: tid=0 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 client.16080 shutdown 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 msgr2=0x5638fc8db100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 0x5638fc8db100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f35d80aa3d0 tx=0x7f35d80b1040 comp rx=0 tx=0).stop 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 shutdown_connections 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638fc861d40 0x5638fc8db640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638fc85d2c0 0x5638fc8dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5638fc8e7010 0x5638fc8e9400 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 --2- 172.21.15.99:0/79736580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638fc861420 0x5638fc8db100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35d17fa700 0 client.16080 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 >> 172.21.15.99:0/79736580 conn(0x5638fc789eb0 msgr2=0x5638fc85bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 shutdown_connections 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 1 -- 172.21.15.99:0/79736580 wait complete. 2024-05-14T17:10:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.986+0000 7f35fa605b80 20 client.16080 trim_cache size 0 max 16384 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 29, 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 20, 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "cbc14f15": { 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 20, 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:10:17.988 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 133.7, 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:17.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:17.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:17.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:17.990 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:10:17.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f35d40b29f0 msgr2=0x7f35d40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f35d40b29f0 0x7f35d40b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35e400abd0 tx=0x7f35e400a380 comp rx=0 tx=0).stop 2024-05-14T17:10:17.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 msgr2=0x7f35e819af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 0x7f35e819af70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d80047b0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 shutdown_connections 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f35d40b29f0 0x7f35d40b4eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35e8108520 0x7f35e81a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102440 0x7f35e819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 --2- 172.21.15.99:0/2561049049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e8102e00 0x7f35e819af70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 >> 172.21.15.99:0/2561049049 conn(0x7f35e80793f0 msgr2=0x7f35e81004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 shutdown_connections 2024-05-14T17:10:17.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:17.987+0000 7f35efd6f700 1 -- 172.21.15.99:0/2561049049 wait complete. 2024-05-14T17:10:18.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 336937 7.12 MB/sec execute 150 sec latency 13128.024 ms 2024-05-14T17:10:18.740 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cbc14f15', total seconds elapsed: 19.1 2024-05-14T17:10:19.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:18 smithi099 ceph-mon[31851]: pgmap v205: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-05-14T17:10:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:18 smithi150 ceph-mon[36204]: pgmap v205: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-05-14T17:10:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:18 smithi193 ceph-mon[35962]: pgmap v205: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.9 MiB/s wr, 94 op/s 2024-05-14T17:10:19.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338587 7.12 MB/sec execute 151 sec latency 26.945 ms 2024-05-14T17:10:20.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338670 7.08 MB/sec execute 152 sec latency 937.882 ms 2024-05-14T17:10:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:20 smithi150 ceph-mon[36204]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.3 MiB/s wr, 112 op/s 2024-05-14T17:10:21.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:20 smithi099 ceph-mon[31851]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.3 MiB/s wr, 112 op/s 2024-05-14T17:10:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:20 smithi193 ceph-mon[35962]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.3 MiB/s wr, 112 op/s 2024-05-14T17:10:21.532 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338670 7.03 MB/sec execute 153 sec latency 1937.950 ms 2024-05-14T17:10:22.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338670 6.98 MB/sec execute 154 sec latency 2938.041 ms 2024-05-14T17:10:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:22 smithi150 ceph-mon[36204]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-05-14T17:10:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:22 smithi099 ceph-mon[31851]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-05-14T17:10:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:22 smithi193 ceph-mon[35962]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-05-14T17:10:23.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338670 6.94 MB/sec execute 155 sec latency 3938.159 ms 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 -- 172.21.15.99:0/480154001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900102330 msgr2=0x7f790010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 --2- 172.21.15.99:0/480154001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900102330 0x7f790010e840 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009cd0 tx=0x7f78f802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 -- 172.21.15.99:0/480154001 shutdown_connections 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 --2- 172.21.15.99:0/480154001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900102330 0x7f790010e840 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 --2- 172.21.15.99:0/480154001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900101970 0x7f7900101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 --2- 172.21.15.99:0/480154001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900107980 0x7f7900107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 -- 172.21.15.99:0/480154001 >> 172.21.15.99:0/480154001 conn(0x7f7900077cd0 msgr2=0x7f79000780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 -- 172.21.15.99:0/480154001 shutdown_connections 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 -- 172.21.15.99:0/480154001 wait complete. 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 Processor -- start 2024-05-14T17:10:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.074+0000 7f7908111640 1 -- start start 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900101970 0x7f7900198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900102330 0x7f7900198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 0x7f790019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7900111c40 con 0x7f7900107980 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7900111ac0 con 0x7f7900102330 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7900111dc0 con 0x7f7900101970 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 0x7f790019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 0x7f790019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35928/0 (socket says 172.21.15.99:35928) 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 -- 172.21.15.99:0/2255641237 learned_addr learned my addr 172.21.15.99:0/2255641237 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7905ead640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900101970 0x7f7900198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79056ac640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900102330 0x7f7900198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 -- 172.21.15.99:0/2255641237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900101970 msgr2=0x7f7900198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900101970 0x7f7900198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 -- 172.21.15.99:0/2255641237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900102330 msgr2=0x7f7900198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900102330 0x7f7900198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 -- 172.21.15.99:0/2255641237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790019d010 con 0x7f7900107980 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7905ead640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900101970 0x7f7900198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79056ac640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900102330 0x7f7900198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f79066ae640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 0x7f790019c8e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f78f8033040 tx=0x7f78f8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f78f6ffd640 1 -- 172.21.15.99:0/2255641237 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8004040 con 0x7f7900107980 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f78f6ffd640 1 -- 172.21.15.99:0/2255641237 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f8004690 con 0x7f7900107980 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f78f6ffd640 1 -- 172.21.15.99:0/2255641237 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8030e20 con 0x7f7900107980 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.075+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790019d2a0 con 0x7f7900107980 2024-05-14T17:10:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.076+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a4b90 con 0x7f7900107980 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f78f6ffd640 1 -- 172.21.15.99:0/2255641237 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f78f80305a0 con 0x7f7900107980 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 Processor -- start 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 -- start start 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff17b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560dbfe8ac30 con 0x560dbff17750 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560dbfe7a7d0 con 0x560dbff24ce0 2024-05-14T17:10:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.078+0000 7f790ae22740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560dbfe7e810 con 0x560dbff18070 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff17b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff17b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35934/0 (socket says 172.21.15.99:35934) 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 -- 172.21.15.99:0/117114367 learned_addr learned my addr 172.21.15.99:0/117114367 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f7905ead640 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f78f6ffd640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f78ec0b2a10 0x7f78ec0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 -- 172.21.15.99:0/117114367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 msgr2=0x560dbff247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 -- 172.21.15.99:0/117114367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 msgr2=0x560dbff270d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff270d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 -- 172.21.15.99:0/117114367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbfb8f210 con 0x560dbff17750 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f7905ead640 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79066ae640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f78ec0b2a10 0x7f78ec0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f79056ac640 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff17b30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005fd0 tx=0x7f78f0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c99b9094e5ba99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f78f6ffd640 1 -- 172.21.15.99:0/2255641237 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f78f802e030 con 0x7f7900107980 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.079+0000 7f78ebfff640 1 -- 172.21.15.99:0/117114367 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00150f0 con 0x560dbff17750 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f78ebfff640 1 -- 172.21.15.99:0/117114367 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f00054d0 con 0x560dbff17750 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f78ebfff640 1 -- 172.21.15.99:0/117114367 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0004d70 con 0x560dbff17750 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 -- 172.21.15.99:0/117114367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 msgr2=0x560dbff17b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f79066ae640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f78ec0b2a10 0x7f78ec0b4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f78f80c8380 tx=0x7f78f80fa720 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff17b30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005fd0 tx=0x7f78f0009670 comp rx=0 tx=0).stop 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 -- 172.21.15.99:0/117114367 shutdown_connections 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff270d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 --2- 172.21.15.99:0/117114367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff17b30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 -- 172.21.15.99:0/117114367 >> 172.21.15.99:0/117114367 conn(0x560dbff13e00 msgr2=0x560dbff141e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 -- 172.21.15.99:0/117114367 shutdown_connections 2024-05-14T17:10:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.080+0000 7f790ae22740 1 -- 172.21.15.99:0/117114367 wait complete. 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 -- start start 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 Processor -- start 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff21e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560dbfb60790 con 0x560dbff18070 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560dbfe36890 con 0x560dbff17750 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560dbff27950 con 0x560dbff24ce0 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f790ae22740 10 client.0 _ll_register_callbacks cb 0x560dbfe39d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f7905ead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f7905ead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35948/0 (socket says 172.21.15.99:35948) 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f7905ead640 1 -- 172.21.15.99:0/2652352078 learned_addr learned my addr 172.21.15.99:0/2652352078 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.081+0000 7f79056ac640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f7905ead640 1 -- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 msgr2=0x560dbff21e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f79056ac640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff21e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f7905ead640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff21e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f7905ead640 1 -- 172.21.15.99:0/2652352078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 msgr2=0x560dbff245f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f7905ead640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff245f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f7905ead640 1 -- 172.21.15.99:0/2652352078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbfc5b620 con 0x560dbff18070 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f79056ac640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff21e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f7905ead640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff218e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f78fc013750 tx=0x7f78fc013c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63635e6697d867 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f78e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f78e97fa640 1 -- 172.21.15.99:0/2652352078 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc020060 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f78e97fa640 1 -- 172.21.15.99:0/2652352078 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f78fc011e30 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f00054d0 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f78e97fa640 1 -- 172.21.15.99:0/2652352078 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78fc013e60 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f790ae22740 10 client.16092 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560dbff33400 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.082+0000 7f78e97fa640 1 -- 172.21.15.99:0/2652352078 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f00054d0 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f78e97fa640 1 -- 172.21.15.99:0/2652352078 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f78fc01a030 con 0x560dbff18070 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f790ae22740 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f790ae22740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f790ae22740 20 client.16092 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f790ae22740 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff34930 0x560dbff36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f790ae22740 4 client.16092 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dbfe8cb80 con 0x560dbff34930 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f79056ac640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff34930 0x560dbff36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f78e97fa640 10 client.16092 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.083+0000 7f79056ac640 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff34930 0x560dbff36d80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffcb5e5f0229f21 server_cookie=517d40942641c0f7 in_seq=0 out_seq=0 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.084+0000 7f78e97fa640 1 -- 172.21.15.99:0/2652352078 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560dbfe8cb80 con 0x560dbff34930 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.084+0000 7f78e97fa640 10 client.16092 handle_command_reply: tid=0 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.084+0000 7f790ae22740 1 client.16092 shutdown 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.084+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 msgr2=0x560dbff218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.084+0000 7f790ae22740 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff218e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f78fc013750 tx=0x7f78fc013c10 comp rx=0 tx=0).stop 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 shutdown_connections 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff34930 0x560dbff36d80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff21e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff218e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f78e97fa640 0 client.16092 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 --2- 172.21.15.99:0/2652352078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff245f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 >> 172.21.15.99:0/2652352078 conn(0x560dbff13e00 msgr2=0x560dbff13100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 shutdown_connections 2024-05-14T17:10:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 1 -- 172.21.15.99:0/2652352078 wait complete. 2024-05-14T17:10:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.085+0000 7f790ae22740 20 client.16092 trim_cache size 0 max 16384 2024-05-14T17:10:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 Processor -- start 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 -- start start 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff30860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff24120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79001aecd0 con 0x560dbff17750 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79001ac540 con 0x560dbff24ce0 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f790ae22740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79001aa3b0 con 0x560dbff18070 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff30860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff30860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35956/0 (socket says 172.21.15.99:35956) 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 -- 172.21.15.99:0/1966671280 learned_addr learned my addr 172.21.15.99:0/1966671280 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f7905ead640 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 -- 172.21.15.99:0/1966671280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 msgr2=0x560dbff208c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff208c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 -- 172.21.15.99:0/1966671280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 msgr2=0x560dbff24120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff24120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.100+0000 7f79056ac640 1 -- 172.21.15.99:0/1966671280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbff13c30 con 0x560dbff17750 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f79056ac640 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff30860 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005d00 tx=0x7f78f0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5493d95c1b20472 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f78eb7fe640 1 -- 172.21.15.99:0/1966671280 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f002f070 con 0x560dbff17750 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f78eb7fe640 1 -- 172.21.15.99:0/1966671280 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f0004680 con 0x560dbff17750 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f78eb7fe640 1 -- 172.21.15.99:0/1966671280 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00104d0 con 0x560dbff17750 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 -- 172.21.15.99:0/1966671280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 msgr2=0x560dbff30860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff30860 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005d00 tx=0x7f78f0004a30 comp rx=0 tx=0).stop 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 -- 172.21.15.99:0/1966671280 shutdown_connections 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff24ce0 0x560dbff24120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff18070 0x560dbff208c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 --2- 172.21.15.99:0/1966671280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff17750 0x560dbff30860 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 -- 172.21.15.99:0/1966671280 >> 172.21.15.99:0/1966671280 conn(0x560dbff13e00 msgr2=0x560dbff201b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 -- 172.21.15.99:0/1966671280 shutdown_connections 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.101+0000 7f790ae22740 1 -- 172.21.15.99:0/1966671280 wait complete. 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 -- start start 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 Processor -- start 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff1e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff1eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff1f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79001aaa00 con 0x560dbff18070 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79001abe70 con 0x560dbff17750 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79001ab0d0 con 0x560dbff24ce0 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f790ae22740 10 client.0 _ll_register_callbacks cb 0x7f78fc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f7905ead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff1eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f7905ead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff1eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35960/0 (socket says 172.21.15.99:35960) 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f7905ead640 1 -- 172.21.15.99:0/3575362205 learned_addr learned my addr 172.21.15.99:0/3575362205 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.102+0000 7f79056ac640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff1e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f7905ead640 1 -- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 msgr2=0x560dbff1f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f79056ac640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff1f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f7905ead640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff1f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f7905ead640 1 -- 172.21.15.99:0/3575362205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 msgr2=0x560dbff1e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f7905ead640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff1e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f7905ead640 1 -- 172.21.15.99:0/3575362205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbff9e250 con 0x560dbff18070 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f79056ac640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff1f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f7905ead640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff1eb20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78fc023670 tx=0x7f78fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02b26d5a15bcfd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f78e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f78e97fa640 1 -- 172.21.15.99:0/3575362205 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc023cf0 con 0x560dbff18070 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f78e97fa640 1 -- 172.21.15.99:0/3575362205 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f78fc005040 con 0x560dbff18070 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f0004680 con 0x560dbff18070 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f78e97fa640 1 -- 172.21.15.99:0/3575362205 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78fc020020 con 0x560dbff18070 2024-05-14T17:10:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f790ae22740 10 client.16098 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560dbff9f3e0 con 0x560dbff18070 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f78e97fa640 1 -- 172.21.15.99:0/3575362205 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f0004680 con 0x560dbff18070 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.103+0000 7f78e97fa640 1 -- 172.21.15.99:0/3575362205 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f78fc01a070 con 0x560dbff18070 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f790ae22740 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f790ae22740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f790ae22740 20 client.16098 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f790ae22740 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff9fae0 0x560dbffa1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f790ae22740 4 client.16098 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dbfe86360 con 0x560dbff9fae0 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f79056ac640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff9fae0 0x560dbffa1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f78e97fa640 10 client.16098 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.104+0000 7f79056ac640 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff9fae0 0x560dbffa1f00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a616f535421fb7e0 server_cookie=ae63741f4d30be9 in_seq=0 out_seq=0 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f78e97fa640 1 -- 172.21.15.99:0/3575362205 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560dbfe86360 con 0x560dbff9fae0 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f78e97fa640 10 client.16098 handle_command_reply: tid=0 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 client.16098 shutdown 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 msgr2=0x560dbff1eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff1eb20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78fc023670 tx=0x7f78fc011810 comp rx=0 tx=0).stop 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 shutdown_connections 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560dbff9fae0 0x560dbffa1f00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dbff24ce0 0x560dbff1f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560dbff18070 0x560dbff1eb20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f78e97fa640 0 client.16098 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 --2- 172.21.15.99:0/3575362205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbff17750 0x560dbff1e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 >> 172.21.15.99:0/3575362205 conn(0x560dbff13e00 msgr2=0x560dbff192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 shutdown_connections 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 1 -- 172.21.15.99:0/3575362205 wait complete. 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.146+0000 7f790ae22740 20 client.16098 trim_cache size 0 max 16384 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:10:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:10:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f78ec0b2a10 msgr2=0x7f78ec0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f78ec0b2a10 0x7f78ec0b4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f78f80c8380 tx=0x7f78f80fa720 comp rx=0 tx=0).stop 2024-05-14T17:10:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 msgr2=0x7f790019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 0x7f790019c8e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f78f8033040 tx=0x7f78f8003970 comp rx=0 tx=0).stop 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 shutdown_connections 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f78ec0b2a10 0x7f78ec0b4ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7900107980 0x7f790019c8e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7900102330 0x7f7900198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 --2- 172.21.15.99:0/2255641237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7900101970 0x7f7900198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 >> 172.21.15.99:0/2255641237 conn(0x7f7900077cd0 msgr2=0x7f79000ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 shutdown_connections 2024-05-14T17:10:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:24.147+0000 7f7908111640 1 -- 172.21.15.99:0/2255641237 wait complete. 2024-05-14T17:10:24.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 338670 6.90 MB/sec execute 156 sec latency 4938.272 ms 2024-05-14T17:10:25.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:24 smithi150 ceph-mon[36204]: pgmap v208: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-05-14T17:10:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:24 smithi099 ceph-mon[31851]: pgmap v208: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-05-14T17:10:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:24 smithi193 ceph-mon[35962]: pgmap v208: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-05-14T17:10:25.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 340894 6.94 MB/sec execute 157 sec latency 4975.722 ms 2024-05-14T17:10:26.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 342210 6.93 MB/sec execute 158 sec latency 291.048 ms 2024-05-14T17:10:27.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:26 smithi150 ceph-mon[36204]: pgmap v209: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-14T17:10:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:26 smithi099 ceph-mon[31851]: pgmap v209: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-14T17:10:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:26 smithi193 ceph-mon[35962]: pgmap v209: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-14T17:10:27.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 342210 6.89 MB/sec execute 159 sec latency 1291.080 ms 2024-05-14T17:10:28.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 342210 6.84 MB/sec execute 160 sec latency 2291.122 ms 2024-05-14T17:10:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:28 smithi150 ceph-mon[36204]: pgmap v210: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-14T17:10:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:28 smithi099 ceph-mon[31851]: pgmap v210: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-14T17:10:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:28 smithi193 ceph-mon[35962]: pgmap v210: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-05-14T17:10:29.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 342210 6.80 MB/sec execute 161 sec latency 3291.221 ms 2024-05-14T17:10:30.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 344435 6.84 MB/sec execute 162 sec latency 3351.714 ms 2024-05-14T17:10:31.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:30 smithi099 ceph-mon[31851]: pgmap v211: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-05-14T17:10:31.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:30 smithi150 ceph-mon[36204]: pgmap v211: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-05-14T17:10:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:30 smithi193 ceph-mon[35962]: pgmap v211: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-05-14T17:10:31.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 345759 6.83 MB/sec execute 163 sec latency 263.211 ms 2024-05-14T17:10:32.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 345759 6.79 MB/sec execute 164 sec latency 1263.316 ms 2024-05-14T17:10:33.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:32 smithi150 ceph-mon[36204]: pgmap v212: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 997 KiB/s wr, 97 op/s 2024-05-14T17:10:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:32 smithi099 ceph-mon[31851]: pgmap v212: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 997 KiB/s wr, 97 op/s 2024-05-14T17:10:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:32 smithi193 ceph-mon[35962]: pgmap v212: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 997 KiB/s wr, 97 op/s 2024-05-14T17:10:33.533 INFO:tasks.workunit.client.0.smithi099.stdout: 1 345759 6.75 MB/sec execute 165 sec latency 2263.433 ms 2024-05-14T17:10:34.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 345759 6.71 MB/sec execute 166 sec latency 3263.523 ms 2024-05-14T17:10:35.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:34 smithi150 ceph-mon[36204]: pgmap v213: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-14T17:10:35.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:34 smithi099 ceph-mon[31851]: pgmap v213: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-14T17:10:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:34 smithi193 ceph-mon[35962]: pgmap v213: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-14T17:10:35.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 348052 6.75 MB/sec execute 167 sec latency 3293.712 ms 2024-05-14T17:10:36.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 349302 6.74 MB/sec execute 168 sec latency 321.676 ms 2024-05-14T17:10:37.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:36 smithi150 ceph-mon[36204]: pgmap v214: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 695 KiB/s wr, 97 op/s 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 -- 172.21.15.99:0/1063875990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460101980 msgr2=0x7fb460101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 --2- 172.21.15.99:0/1063875990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460101980 0x7fb460101e00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb450009930 tx=0x7fb45002a420 comp rx=0 tx=0).stop 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 -- 172.21.15.99:0/1063875990 shutdown_connections 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 --2- 172.21.15.99:0/1063875990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460102340 0x7fb46010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 --2- 172.21.15.99:0/1063875990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460101980 0x7fb460101e00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 --2- 172.21.15.99:0/1063875990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460107940 0x7fb460107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.236+0000 7fb4682bb640 1 -- 172.21.15.99:0/1063875990 >> 172.21.15.99:0/1063875990 conn(0x7fb460078ad0 msgr2=0x7fb460078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 -- 172.21.15.99:0/1063875990 shutdown_connections 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 -- 172.21.15.99:0/1063875990 wait complete. 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 Processor -- start 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 -- start start 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460101980 0x7fb46019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460102340 0x7fb46019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 0x7fb4601a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb460111c00 con 0x7fb460107940 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb460111a80 con 0x7fb460101980 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb4682bb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb460111d80 con 0x7fb460102340 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb466858640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 0x7fb4601a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb466858640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 0x7fb4601a50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43452/0 (socket says 172.21.15.99:43452) 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb466858640 1 -- 172.21.15.99:0/3100761577 learned_addr learned my addr 172.21.15.99:0/3100761577 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.237+0000 7fb465856640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460102340 0x7fb46019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466057640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460101980 0x7fb46019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466858640 1 -- 172.21.15.99:0/3100761577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460102340 msgr2=0x7fb46019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466858640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460102340 0x7fb46019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466858640 1 -- 172.21.15.99:0/3100761577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460101980 msgr2=0x7fb46019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466858640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460101980 0x7fb46019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466858640 1 -- 172.21.15.99:0/3100761577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4601a5620 con 0x7fb460107940 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466057640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460101980 0x7fb46019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb466858640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 0x7fb4601a50e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb45800eb10 tx=0x7fb45800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb4577fe640 1 -- 172.21.15.99:0/3100761577 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458004b20 con 0x7fb460107940 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb4577fe640 1 -- 172.21.15.99:0/3100761577 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb458004cc0 con 0x7fb460107940 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb4577fe640 1 -- 172.21.15.99:0/3100761577 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458010670 con 0x7fb460107940 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4601a57f0 con 0x7fb460107940 2024-05-14T17:10:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.238+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4601a5d00 con 0x7fb460107940 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb4577fe640 1 -- 172.21.15.99:0/3100761577 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb458005020 con 0x7fb460107940 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 Processor -- start 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 -- start start 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c925c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c9349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558a4c898c30 con 0x558a4c934ee0 2024-05-14T17:10:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558a4c8887d0 con 0x558a4c925830 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb46afcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a4c88c810 con 0x558a4c926150 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb465856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb465856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9372d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43456/0 (socket says 172.21.15.99:43456) 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.241+0000 7fb465856640 1 -- 172.21.15.99:0/3810695061 learned_addr learned my addr 172.21.15.99:0/3810695061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c925c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb466057640 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c9349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb4577fe640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb44c0b2a90 0x7fb44c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 -- 172.21.15.99:0/3810695061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 msgr2=0x558a4c9349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c9349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 -- 172.21.15.99:0/3810695061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 msgr2=0x558a4c925c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c925c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 -- 172.21.15.99:0/3810695061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4c59d210 con 0x558a4c934ee0 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb466057640 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c9349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb466858640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb44c0b2a90 0x7fb44c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb465856640 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9372d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb450009930 tx=0x7fb450005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d7086a1177f6c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb4577fe640 1 -- 172.21.15.99:0/3100761577 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb4580e2b00 con 0x7fb460107940 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb454bfe640 1 -- 172.21.15.99:0/3810695061 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450035070 con 0x558a4c934ee0 2024-05-14T17:10:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb454bfe640 1 -- 172.21.15.99:0/3810695061 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb450004030 con 0x558a4c934ee0 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.242+0000 7fb454bfe640 1 -- 172.21.15.99:0/3810695061 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4500052d0 con 0x558a4c934ee0 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 -- 172.21.15.99:0/3810695061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 msgr2=0x558a4c9372d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9372d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb450009930 tx=0x7fb450005fd0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb466858640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb44c0b2a90 0x7fb44c0b4f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb4580a9ae0 tx=0x7fb45807f040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 -- 172.21.15.99:0/3810695061 shutdown_connections 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9372d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c9349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 --2- 172.21.15.99:0/3810695061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c925c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 -- 172.21.15.99:0/3810695061 >> 172.21.15.99:0/3810695061 conn(0x558a4c921e00 msgr2=0x558a4c9221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 -- 172.21.15.99:0/3810695061 shutdown_connections 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 -- 172.21.15.99:0/3810695061 wait complete. 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 -- start start 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.243+0000 7fb46afcd740 1 Processor -- start 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c934870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c931b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558a4c56e790 con 0x558a4c934ee0 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558a4c844890 con 0x558a4c925830 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a4c937a90 con 0x558a4c926150 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 10 client.0 _ll_register_callbacks cb 0x558a4c847d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9320e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43464/0 (socket says 172.21.15.99:43464) 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 -- 172.21.15.99:0/2763087607 learned_addr learned my addr 172.21.15.99:0/2763087607 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb466057640 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c931b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 -- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 msgr2=0x558a4c931b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c931b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 -- 172.21.15.99:0/2763087607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 msgr2=0x558a4c934870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c934870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 -- 172.21.15.99:0/2763087607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4c669620 con 0x558a4c934ee0 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb465856640 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9320e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb45002a940 tx=0x7fb45002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1d1cd21a2b737c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb449ffb640 1 -- 172.21.15.99:0/2763087607 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450035050 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb449ffb640 1 -- 172.21.15.99:0/2763087607 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb450031b10 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.244+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb450004030 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb449ffb640 1 -- 172.21.15.99:0/2763087607 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb45005a050 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a4c943580 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb449ffb640 1 -- 172.21.15.99:0/2763087607 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb450004030 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb449ffb640 1 -- 172.21.15.99:0/2763087607 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4500049e0 con 0x558a4c934ee0 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 20 client.16110 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9448d0 0x558a4c946cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 4 client.16110 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a4c89ab80 con 0x558a4c9448d0 2024-05-14T17:10:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.245+0000 7fb466057640 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9448d0 0x558a4c946cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.246+0000 7fb449ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.246+0000 7fb466057640 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9448d0 0x558a4c946cc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416dd74c1eed69ae server_cookie=230b52aa6c938a62 in_seq=0 out_seq=0 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.246+0000 7fb449ffb640 1 -- 172.21.15.99:0/2763087607 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a4c89ab80 con 0x558a4c9448d0 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.246+0000 7fb449ffb640 10 client.16110 handle_command_reply: tid=0 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.246+0000 7fb46afcd740 1 client.16110 shutdown 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 msgr2=0x558a4c9320e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9320e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb45002a940 tx=0x7fb45002a420 comp rx=0 tx=0).stop 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 shutdown_connections 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9448d0 0x558a4c946cc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c934ee0 0x558a4c9320e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb449ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c926150 0x558a4c931b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2763087607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c925830 0x558a4c934870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 >> 172.21.15.99:0/2763087607 conn(0x558a4c921e00 msgr2=0x558a4c921100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 shutdown_connections 2024-05-14T17:10:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 1 -- 172.21.15.99:0/2763087607 wait complete. 2024-05-14T17:10:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.247+0000 7fb46afcd740 20 client.16110 trim_cache size 0 max 16384 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 Processor -- start 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 -- start start 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 0x558a4c9343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c926150 0x558a4c949ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c934ee0 0x558a4c94a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601aedb0 con 0x558a4c925830 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601ac620 con 0x558a4c926150 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb46afcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601aa490 con 0x558a4c934ee0 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 0x558a4c9343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 0x558a4c9343a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43478/0 (socket says 172.21.15.99:43478) 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 -- 172.21.15.99:0/2734868488 learned_addr learned my addr 172.21.15.99:0/2734868488 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 -- 172.21.15.99:0/2734868488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c934ee0 msgr2=0x558a4c94a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb466057640 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c926150 0x558a4c949ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c934ee0 0x558a4c94a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 -- 172.21.15.99:0/2734868488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c926150 msgr2=0x558a4c949ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c926150 0x558a4c949ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 -- 172.21.15.99:0/2734868488 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4c925f80 con 0x558a4c925830 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb466057640 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c926150 0x558a4c949ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb465856640 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 0x558a4c9343a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb450001eb0 tx=0x7fb4500315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e1e38bf8bcbd70 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb44bfff640 1 -- 172.21.15.99:0/2734868488 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450035070 con 0x558a4c925830 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb44bfff640 1 -- 172.21.15.99:0/2734868488 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb450051070 con 0x558a4c925830 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.262+0000 7fb44bfff640 1 -- 172.21.15.99:0/2734868488 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4500039b0 con 0x558a4c925830 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 -- 172.21.15.99:0/2734868488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 msgr2=0x558a4c9343a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 0x558a4c9343a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb450001eb0 tx=0x7fb4500315a0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 -- 172.21.15.99:0/2734868488 shutdown_connections 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c934ee0 0x558a4c94a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c926150 0x558a4c949ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 --2- 172.21.15.99:0/2734868488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c925830 0x558a4c9343a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 -- 172.21.15.99:0/2734868488 >> 172.21.15.99:0/2734868488 conn(0x558a4c921e00 msgr2=0x558a4c92d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 -- 172.21.15.99:0/2734868488 shutdown_connections 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 -- 172.21.15.99:0/2734868488 wait complete. 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 -- start start 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.263+0000 7fb46afcd740 1 Processor -- start 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c925830 0x558a4c94a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 0x558a4c947740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c934ee0 0x558a4c947e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601aaae0 con 0x558a4c926150 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601abf50 con 0x558a4c934ee0 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601ab1b0 con 0x558a4c925830 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 10 client.0 _ll_register_callbacks cb 0x7fb450009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 0x558a4c947740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 0x558a4c947740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43494/0 (socket says 172.21.15.99:43494) 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 -- 172.21.15.99:0/1127585080 learned_addr learned my addr 172.21.15.99:0/1127585080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb465856640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c925830 0x558a4c94a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 -- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c925830 msgr2=0x558a4c94a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c925830 0x558a4c94a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 -- 172.21.15.99:0/1127585080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c934ee0 msgr2=0x558a4c947e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c934ee0 0x558a4c947e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 -- 172.21.15.99:0/1127585080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4c9addf0 con 0x558a4c926150 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb465856640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c925830 0x558a4c94a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb466057640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 0x558a4c947740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x558a4c937ba0 tx=0x7fb45c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4950290236293260 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:37.267 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:36 smithi099 ceph-mon[31851]: pgmap v214: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 695 KiB/s wr, 97 op/s 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb449ffb640 1 -- 172.21.15.99:0/1127585080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01b640 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb449ffb640 1 -- 172.21.15.99:0/1127585080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb45c01b7e0 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.264+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb450051070 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb449ffb640 1 -- 172.21.15.99:0/1127585080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb45c01b9e0 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a4c9aef80 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb449ffb640 1 -- 172.21.15.99:0/1127585080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb450051070 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb449ffb640 1 -- 172.21.15.99:0/1127585080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb45c00f040 con 0x558a4c926150 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 20 client.16116 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9af790 0x558a4c9b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 4 client.16116 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a4c894360 con 0x558a4c9af790 2024-05-14T17:10:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.265+0000 7fb465856640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9af790 0x558a4c9b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.266+0000 7fb449ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.266+0000 7fb465856640 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9af790 0x558a4c9b1bb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8d6c7dcb679c60 server_cookie=a2dbbe7105fb93e8 in_seq=0 out_seq=0 2024-05-14T17:10:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:36 smithi193 ceph-mon[35962]: pgmap v214: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 695 KiB/s wr, 97 op/s 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb449ffb640 1 -- 172.21.15.99:0/1127585080 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a4c894360 con 0x558a4c9af790 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb449ffb640 10 client.16116 handle_command_reply: tid=0 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 client.16116 shutdown 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 msgr2=0x558a4c947740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 0x558a4c947740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x558a4c937ba0 tx=0x7fb45c012820 comp rx=0 tx=0).stop 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 shutdown_connections 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558a4c9af790 0x558a4c9b1bb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558a4c934ee0 0x558a4c947e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a4c926150 0x558a4c947740 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb449ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 --2- 172.21.15.99:0/1127585080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a4c925830 0x558a4c94a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 >> 172.21.15.99:0/1127585080 conn(0x558a4c921e00 msgr2=0x558a4c927280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 shutdown_connections 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 1 -- 172.21.15.99:0/1127585080 wait complete. 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.302+0000 7fb46afcd740 20 client.16116 trim_cache size 0 max 16384 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:10:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb44c0b2a90 msgr2=0x7fb44c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb44c0b2a90 0x7fb44c0b4f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb4580a9ae0 tx=0x7fb45807f040 comp rx=0 tx=0).stop 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 msgr2=0x7fb4601a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 0x7fb4601a50e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb45800eb10 tx=0x7fb45800c2d0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 shutdown_connections 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb44c0b2a90 0x7fb44c0b4f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460107940 0x7fb4601a50e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460102340 0x7fb46019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 --2- 172.21.15.99:0/3100761577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb460101980 0x7fb46019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 >> 172.21.15.99:0/3100761577 conn(0x7fb460078ad0 msgr2=0x7fb460104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 shutdown_connections 2024-05-14T17:10:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:37.303+0000 7fb4682bb640 1 -- 172.21.15.99:0/3100761577 wait complete. 2024-05-14T17:10:37.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 349302 6.70 MB/sec execute 169 sec latency 1321.702 ms 2024-05-14T17:10:38.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 349302 6.66 MB/sec execute 170 sec latency 2321.777 ms 2024-05-14T17:10:39.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:38 smithi150 ceph-mon[36204]: pgmap v215: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 695 KiB/s wr, 97 op/s 2024-05-14T17:10:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:38 smithi099 ceph-mon[31851]: pgmap v215: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 695 KiB/s wr, 97 op/s 2024-05-14T17:10:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:38 smithi193 ceph-mon[35962]: pgmap v215: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 695 KiB/s wr, 97 op/s 2024-05-14T17:10:39.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 349302 6.63 MB/sec execute 171 sec latency 3321.871 ms 2024-05-14T17:10:40.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 351546 6.67 MB/sec execute 172 sec latency 3381.908 ms 2024-05-14T17:10:41.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:40 smithi150 ceph-mon[36204]: pgmap v216: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1022 KiB/s wr, 153 op/s 2024-05-14T17:10:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:40 smithi099 ceph-mon[31851]: pgmap v216: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1022 KiB/s wr, 153 op/s 2024-05-14T17:10:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:40 smithi193 ceph-mon[35962]: pgmap v216: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1022 KiB/s wr, 153 op/s 2024-05-14T17:10:41.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 352850 6.66 MB/sec execute 173 sec latency 147.532 ms 2024-05-14T17:10:42.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 352850 6.62 MB/sec execute 174 sec latency 1147.647 ms 2024-05-14T17:10:43.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:42 smithi150 ceph-mon[36204]: pgmap v217: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 678 KiB/s wr, 97 op/s 2024-05-14T17:10:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:42 smithi099 ceph-mon[31851]: pgmap v217: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 678 KiB/s wr, 97 op/s 2024-05-14T17:10:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:42 smithi193 ceph-mon[35962]: pgmap v217: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 678 KiB/s wr, 97 op/s 2024-05-14T17:10:43.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 352850 6.58 MB/sec execute 175 sec latency 2147.707 ms 2024-05-14T17:10:44.184 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.4 2024-05-14T17:10:44.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:10:44.439 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:10:44.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 352850 6.55 MB/sec execute 176 sec latency 3147.767 ms 2024-05-14T17:10:45.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:44 smithi150 ceph-mon[36204]: pgmap v218: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 960 KiB/s wr, 136 op/s 2024-05-14T17:10:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:44 smithi099 ceph-mon[31851]: pgmap v218: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 960 KiB/s wr, 136 op/s 2024-05-14T17:10:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:44 smithi193 ceph-mon[35962]: pgmap v218: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 960 KiB/s wr, 136 op/s 2024-05-14T17:10:45.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 355119 6.59 MB/sec execute 177 sec latency 3208.540 ms 2024-05-14T17:10:46.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 -- 172.21.15.99:0/2716130641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 msgr2=0x7f2cd0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/2716130641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd0103a60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0099c0 tx=0x7f2cbc009cd0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 -- 172.21.15.99:0/2716130641 shutdown_connections 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/2716130641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0103fa0 0x7f2cd0110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/2716130641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0109680 0x7f2cd0109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/2716130641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd0103a60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 -- 172.21.15.99:0/2716130641 >> 172.21.15.99:0/2716130641 conn(0x7f2cd007a6f0 msgr2=0x7f2cd007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 -- 172.21.15.99:0/2716130641 shutdown_connections 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.327+0000 7f2cd6d95700 1 -- 172.21.15.99:0/2716130641 wait complete. 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 Processor -- start 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 -- start start 2024-05-14T17:10:46.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 0x7f2cd019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0109680 0x7f2cd01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01137e0 con 0x7f2cd01035e0 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0113660 con 0x7f2cd0103fa0 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd6d95700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0113960 con 0x7f2cd0109680 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:46.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 0x7f2cd019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd5332700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0109680 0x7f2cd01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56902/0 (socket says 172.21.15.99:56902) 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 0x7f2cd019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:47778/0 (socket says 172.21.15.99:47778) 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 -- 172.21.15.99:0/420208342 learned_addr learned my addr 172.21.15.99:0/420208342 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 -- 172.21.15.99:0/420208342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0109680 msgr2=0x7f2cd01999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0109680 0x7f2cd01999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 -- 172.21.15.99:0/420208342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 msgr2=0x7f2cd019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 0x7f2cd019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd4b31700 1 -- 172.21.15.99:0/420208342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd019a140 con 0x7f2cd01035e0 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2cd5332700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0109680 0x7f2cd01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:46.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.328+0000 7f2ccffff700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 0x7f2cd019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.329+0000 7f2cd4b31700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd019f820 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e9a0 tx=0x7f2cc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.329+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ca10 con 0x7f2cd01035e0 2024-05-14T17:10:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.329+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd019a430 con 0x7f2cd01035e0 2024-05-14T17:10:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.329+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc400cbb0 con 0x7f2cd01035e0 2024-05-14T17:10:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.329+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4010e60 con 0x7f2cd01035e0 2024-05-14T17:10:46.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.329+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd0075770 con 0x7f2cd01035e0 2024-05-14T17:10:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.330+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd0104cf0 con 0x7f2cd01035e0 2024-05-14T17:10:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.331+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2cc4010430 con 0x7f2cd01035e0 2024-05-14T17:10:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.332+0000 7f2ccdffb700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2cc00b29f0 0x7f2cc00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.332+0000 7f2ccffff700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2cc00b29f0 0x7f2cc00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.332+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f2cc40e2c60 con 0x7f2cd01035e0 2024-05-14T17:10:46.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.332+0000 7f2ccffff700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2cc00b29f0 0x7f2cc00b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2cd007acf0 tx=0x7f2cbc005fb0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:46.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.336+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cc40aa640 con 0x7f2cd01035e0 2024-05-14T17:10:46.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356392 6.58 MB/sec execute 178 sec latency 226.398 ms 2024-05-14T17:10:46.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.605+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd019af10 con 0x7f2cd01035e0 2024-05-14T17:10:46.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.606+0000 7f2ccdffb700 1 -- 172.21.15.99:0/420208342 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11879 (secure 0 0 0) 0x7f2cc400cd60 con 0x7f2cd01035e0 2024-05-14T17:10:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:10:46.606 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:10:04.586724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:10:46.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2cc00b29f0 msgr2=0x7f2cc00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2cc00b29f0 0x7f2cc00b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2cd007acf0 tx=0x7f2cbc005fb0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 msgr2=0x7f2cd019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd019f820 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e9a0 tx=0x7f2cc400ecb0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 shutdown_connections 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2cc00b29f0 0x7f2cc00b4eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd0109680 0x7f2cd01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd01035e0 0x7f2cd019f820 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 --2- 172.21.15.99:0/420208342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd0103fa0 0x7f2cd019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 >> 172.21.15.99:0/420208342 conn(0x7f2cd007a6f0 msgr2=0x7f2cd0102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 shutdown_connections 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:46.609+0000 7f2cd6d95700 1 -- 172.21.15.99:0/420208342 wait complete. 2024-05-14T17:10:46.609 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-05-14T17:10:47.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:46 smithi150 ceph-mon[36204]: pgmap v219: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 609 KiB/s wr, 94 op/s 2024-05-14T17:10:47.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:46 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/420208342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:47.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:46 smithi099 ceph-mon[31851]: pgmap v219: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 609 KiB/s wr, 94 op/s 2024-05-14T17:10:47.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:46 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/420208342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:47.265 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0' 2024-05-14T17:10:47.265 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0 2024-05-14T17:10:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:46 smithi193 ceph-mon[35962]: pgmap v219: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 609 KiB/s wr, 94 op/s 2024-05-14T17:10:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:46 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/420208342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:10:47.517 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:10:47.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356392 6.54 MB/sec execute 179 sec latency 1226.428 ms 2024-05-14T17:10:48.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356392 6.51 MB/sec execute 180 sec latency 2226.508 ms 2024-05-14T17:10:49.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:48 smithi099 ceph-mon[31851]: pgmap v220: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 609 KiB/s wr, 94 op/s 2024-05-14T17:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 -- 172.21.15.99:0/333569806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41241027c0 msgr2=0x7f4124106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 --2- 172.21.15.99:0/333569806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41241027c0 0x7f4124106810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f410c0099c0 tx=0x7f410c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 -- 172.21.15.99:0/333569806 shutdown_connections 2024-05-14T17:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 --2- 172.21.15.99:0/333569806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4124106ef0 0x7f412410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 --2- 172.21.15.99:0/333569806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124101e10 0x7f41241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 --2- 172.21.15.99:0/333569806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41241027c0 0x7f4124106810 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 -- 172.21.15.99:0/333569806 >> 172.21.15.99:0/333569806 conn(0x7f4124078ae0 msgr2=0x7f41240ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 -- 172.21.15.99:0/333569806 shutdown_connections 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.147+0000 7f412a428700 1 -- 172.21.15.99:0/333569806 wait complete. 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 Processor -- start 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 -- start start 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 0x7f412419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41241027c0 0x7f412419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124106ef0 0x7f41241a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4124111df0 con 0x7f4124101e10 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4124111c70 con 0x7f4124106ef0 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f412a428700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4124111f70 con 0x7f41241027c0 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f4123fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 0x7f412419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f4123fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 0x7f412419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56914/0 (socket says 172.21.15.99:56914) 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f4123fff700 1 -- 172.21.15.99:0/437083358 learned_addr learned my addr 172.21.15.99:0/437083358 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f41289c5700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124106ef0 0x7f41241a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.148+0000 7f411bfff700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41241027c0 0x7f412419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4123fff700 1 -- 172.21.15.99:0/437083358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41241027c0 msgr2=0x7f412419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4123fff700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41241027c0 0x7f412419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4123fff700 1 -- 172.21.15.99:0/437083358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124106ef0 msgr2=0x7f41241a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4123fff700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124106ef0 0x7f41241a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4123fff700 1 -- 172.21.15.99:0/437083358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41241a5fa0 con 0x7f4124101e10 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f41289c5700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124106ef0 0x7f41241a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4123fff700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 0x7f412419a9e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f411400b530 tx=0x7f411400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4121ffb700 1 -- 172.21.15.99:0/437083358 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114015020 con 0x7f4124101e10 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4121ffb700 1 -- 172.21.15.99:0/437083358 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4114011ce0 con 0x7f4124101e10 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41241a61d0 con 0x7f4124101e10 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f4121ffb700 1 -- 172.21.15.99:0/437083358 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114010d20 con 0x7f4124101e10 2024-05-14T17:10:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.149+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a66e0 con 0x7f4124101e10 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.151+0000 7f4121ffb700 1 -- 172.21.15.99:0/437083358 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4114018020 con 0x7f4124101e10 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.152+0000 7f4121ffb700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f41100b2aa0 0x7f41100b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.152+0000 7f411bfff700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f41100b2aa0 0x7f41100b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.152+0000 7f4121ffb700 1 -- 172.21.15.99:0/437083358 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f41140e1d80 con 0x7f4124101e10 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f411bfff700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f41100b2aa0 0x7f41100b4f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f410c000c00 tx=0x7f410c01a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 Processor -- start 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 -- start start 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f2d8a0 0x5578d6f2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2e1c0 0x5578d6f29180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5578d6e82f00 con 0x5578d6f29810 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5578d6e72aa0 con 0x5578d6f2d8a0 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5578d6e76ae0 con 0x5578d6f2e1c0 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f357a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56916/0 (socket says 172.21.15.99:56916) 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 -- 172.21.15.99:0/2817057197 learned_addr learned my addr 172.21.15.99:0/2817057197 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4123fff700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f2d8a0 0x5578d6f2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f411bfff700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2e1c0 0x5578d6f29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 -- 172.21.15.99:0/2817057197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2e1c0 msgr2=0x5578d6f29180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2e1c0 0x5578d6f29180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 -- 172.21.15.99:0/2817057197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f2d8a0 msgr2=0x5578d6f2dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f2d8a0 0x5578d6f2dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f41289c5700 1 -- 172.21.15.99:0/2817057197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d6e1c140 con 0x5578d6f29810 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f4123fff700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f2d8a0 0x5578d6f2dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.153+0000 7f411bfff700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2e1c0 0x5578d6f29180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.154+0000 7f41289c5700 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f357a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e230 tx=0x7f411c00e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee43710162563578 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.154+0000 7f411a7fc700 1 -- 172.21.15.99:0/2817057197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c00ee80 con 0x5578d6f29810 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.154+0000 7f411a7fc700 1 -- 172.21.15.99:0/2817057197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c004bb0 con 0x5578d6f29810 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.154+0000 7f411a7fc700 1 -- 172.21.15.99:0/2817057197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c005020 con 0x5578d6f29810 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.154+0000 7f4134cbeb80 1 -- 172.21.15.99:0/2817057197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 msgr2=0x5578d6f357a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.154+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f357a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e230 tx=0x7f411c00e540 comp rx=0 tx=0).stop 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 -- 172.21.15.99:0/2817057197 shutdown_connections 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f357a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f2d8a0 0x5578d6f2dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/2817057197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2e1c0 0x5578d6f29180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 -- 172.21.15.99:0/2817057197 >> 172.21.15.99:0/2817057197 conn(0x5578d6e51f40 msgr2=0x5578d6f1f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 -- 172.21.15.99:0/2817057197 shutdown_connections 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 -- 172.21.15.99:0/2817057197 wait complete. 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 -- start start 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 Processor -- start 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f35350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f32c30 0x5578d6f330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.155+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5578d6d79130 con 0x5578d6f29810 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5578d6ddaf30 con 0x5578d6f32c30 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5578d6f35e30 con 0x5578d6f2d8a0 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4134cbeb80 10 client.0 _ll_register_callbacks cb 0x5578d6e0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f35350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f35350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56930/0 (socket says 172.21.15.99:56930) 2024-05-14T17:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 -- 172.21.15.99:0/1389298147 learned_addr learned my addr 172.21.15.99:0/1389298147 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f411bfff700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f41289c5700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f32c30 0x5578d6f330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 -- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 msgr2=0x5578d6f326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 -- 172.21.15.99:0/1389298147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f32c30 msgr2=0x5578d6f330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f32c30 0x5578d6f330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 -- 172.21.15.99:0/1389298147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d6dc2a60 con 0x5578d6f29810 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4123fff700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f35350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f41140a51f0 tx=0x7f4114010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f938b86810c7a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f41037fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f41037fe700 1 -- 172.21.15.99:0/1389298147 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140c3060 con 0x5578d6f29810 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f41037fe700 1 -- 172.21.15.99:0/1389298147 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f41140b1d50 con 0x5578d6f29810 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f411c004bb0 con 0x5578d6f29810 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f41037fe700 1 -- 172.21.15.99:0/1389298147 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f41140ca050 con 0x5578d6f29810 2024-05-14T17:10:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4134cbeb80 10 client.16134 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.156+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d6e2c8f0 con 0x5578d6f29810 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f41037fe700 1 -- 172.21.15.99:0/1389298147 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c004bb0 con 0x5578d6f29810 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f41037fe700 1 -- 172.21.15.99:0/1389298147 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f41140ad170 con 0x5578d6f29810 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f4134cbeb80 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f4134cbeb80 10 client.16134 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f4134cbeb80 20 client.16134 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6f42bf0 0x5578d6f45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f4134cbeb80 4 client.16134 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d6e84e50 con 0x5578d6f42bf0 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.157+0000 7f411bfff700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6f42bf0 0x5578d6f45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f41037fe700 10 client.16134 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f411bfff700 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6f42bf0 0x5578d6f45020 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c663b7246389d6 server_cookie=afff7f929e265e9 in_seq=0 out_seq=0 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f41037fe700 1 -- 172.21.15.99:0/1389298147 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578d6e84e50 con 0x5578d6f42bf0 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f41037fe700 10 client.16134 handle_command_reply: tid=0 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f4134cbeb80 1 client.16134 shutdown 2024-05-14T17:10:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 msgr2=0x5578d6f35350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.158+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f35350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f41140a51f0 tx=0x7f4114010f70 comp rx=0 tx=0).stop 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 shutdown_connections 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f32c30 0x5578d6f330b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6f42bf0 0x5578d6f45020 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f29810 0x5578d6f35350 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f41037fe700 0 client.16134 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/1389298147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 >> 172.21.15.99:0/1389298147 conn(0x5578d6e51f40 msgr2=0x5578d6f2ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 shutdown_connections 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 1 -- 172.21.15.99:0/1389298147 wait complete. 2024-05-14T17:10:49.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.159+0000 7f4134cbeb80 20 client.16134 trim_cache size 0 max 16384 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 Processor -- start 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 -- start start 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f31220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f34400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f34940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ae510 con 0x5578d6f32c30 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41241abd40 con 0x5578d6f29810 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.178+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41241a9bb0 con 0x5578d6f2d8a0 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f34940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f34940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56940/0 (socket says 172.21.15.99:56940) 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 -- 172.21.15.99:0/881376953 learned_addr learned my addr 172.21.15.99:0/881376953 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f411bfff700 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f34400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f4123fff700 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f31220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 -- 172.21.15.99:0/881376953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 msgr2=0x5578d6f34400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f34400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 -- 172.21.15.99:0/881376953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 msgr2=0x5578d6f31220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f31220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 -- 172.21.15.99:0/881376953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d6f31da0 con 0x5578d6f32c30 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f41289c5700 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f34940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e1a0 tx=0x7f411c007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145c692f2cc691d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f4119ffb700 1 -- 172.21.15.99:0/881376953 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c022070 con 0x5578d6f32c30 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f4119ffb700 1 -- 172.21.15.99:0/881376953 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c0050b0 con 0x5578d6f32c30 2024-05-14T17:10:49.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.179+0000 7f4119ffb700 1 -- 172.21.15.99:0/881376953 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c014480 con 0x5578d6f32c30 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 -- 172.21.15.99:0/881376953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 msgr2=0x5578d6f34940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f34940 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e1a0 tx=0x7f411c007550 comp rx=0 tx=0).stop 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 -- 172.21.15.99:0/881376953 shutdown_connections 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f34940 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f31220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/881376953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f34400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 -- 172.21.15.99:0/881376953 >> 172.21.15.99:0/881376953 conn(0x5578d6e51f40 msgr2=0x5578d6f33cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 -- 172.21.15.99:0/881376953 shutdown_connections 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 -- 172.21.15.99:0/881376953 wait complete. 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 -- start start 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.180+0000 7f4134cbeb80 1 Processor -- start 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f3b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f38960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f38ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41241aa200 con 0x5578d6f32c30 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ab670 con 0x5578d6f29810 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41241aa8d0 con 0x5578d6f2d8a0 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 10 client.0 _ll_register_callbacks cb 0x5578d6e0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f38ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f38ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56944/0 (socket says 172.21.15.99:56944) 2024-05-14T17:10:49.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 -- 172.21.15.99:0/3798501402 learned_addr learned my addr 172.21.15.99:0/3798501402 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f411bfff700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f38960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 -- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 msgr2=0x5578d6f38960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4123fff700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f3b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f38960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 -- 172.21.15.99:0/3798501402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 msgr2=0x5578d6f3b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f3b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 -- 172.21.15.99:0/3798501402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d6f39690 con 0x5578d6f32c30 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f411bfff700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f38960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41289c5700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f38ea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f411c007890 tx=0x7f411c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40833373c9800ff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41037fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41037fe700 1 -- 172.21.15.99:0/3798501402 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c02a070 con 0x5578d6f32c30 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f41037fe700 1 -- 172.21.15.99:0/3798501402 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f411c005470 con 0x5578d6f32c30 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.181+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f411c0050b0 con 0x5578d6f32c30 2024-05-14T17:10:49.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f41037fe700 1 -- 172.21.15.99:0/3798501402 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f411c022070 con 0x5578d6f32c30 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 10 client.16140 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d6faf470 con 0x5578d6f32c30 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f41037fe700 1 -- 172.21.15.99:0/3798501402 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c0050b0 con 0x5578d6f32c30 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f41037fe700 1 -- 172.21.15.99:0/3798501402 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f411c01e450 con 0x5578d6f32c30 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 10 client.16140 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 20 client.16140 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6faf8f0 0x5578d6fb1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 4 client.16140 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}] 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}) -- 0x5578d6e7e630 con 0x5578d6faf8f0 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.182+0000 7f4123fff700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6faf8f0 0x5578d6fb1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.183+0000 7f4123fff700 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6faf8f0 0x5578d6fb1d10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff15fc06fc5646d0 server_cookie=d2dd08ab0f576b36 in_seq=0 out_seq=0 2024-05-14T17:10:49.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.183+0000 7f41037fe700 10 client.16140 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:49.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:48 smithi150 ceph-mon[36204]: pgmap v220: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 609 KiB/s wr, 94 op/s 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f41037fe700 1 -- 172.21.15.99:0/3798501402 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5578d6e7e630 con 0x5578d6faf8f0 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f41037fe700 10 client.16140 handle_command_reply: tid=0 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 client.16140 shutdown 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 msgr2=0x5578d6f38ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f38ea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f411c007890 tx=0x7f411c003ce0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 shutdown_connections 2024-05-14T17:10:49.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5578d6faf8f0 0x5578d6fb1d10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578d6f32c30 0x5578d6f38ea0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578d6f29810 0x5578d6f3b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f41037fe700 0 client.16140 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 --2- 172.21.15.99:0/3798501402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5578d6f2d8a0 0x5578d6f38960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 >> 172.21.15.99:0/3798501402 conn(0x5578d6e51f40 msgr2=0x5578d6f34580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 shutdown_connections 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 1 -- 172.21.15.99:0/3798501402 wait complete. 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.259+0000 7f4134cbeb80 20 client.16140 trim_cache size 0 max 16384 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:10:49.260 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 29, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 22, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "379b651a": { 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 22, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 136.4, 2024-05-14T17:10:49.261 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:10:49.262 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f41100b2aa0 msgr2=0x7f41100b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f41100b2aa0 0x7f41100b4f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f410c000c00 tx=0x7f410c01a040 comp rx=0 tx=0).stop 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 msgr2=0x7f412419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 0x7f412419a9e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f411400b530 tx=0x7f411400b8f0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 shutdown_connections 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f41100b2aa0 0x7f41100b4f60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4124106ef0 0x7f41241a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4124101e10 0x7f412419a9e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 --2- 172.21.15.99:0/437083358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41241027c0 0x7f412419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 >> 172.21.15.99:0/437083358 conn(0x7f4124078ae0 msgr2=0x7f4124106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 shutdown_connections 2024-05-14T17:10:49.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:10:49.260+0000 7f412a428700 1 -- 172.21.15.99:0/437083358 wait complete. 2024-05-14T17:10:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:48 smithi193 ceph-mon[35962]: pgmap v220: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 609 KiB/s wr, 94 op/s 2024-05-14T17:10:49.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.47 MB/sec execute 181 sec latency 2909.730 ms 2024-05-14T17:10:49.991 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '379b651a', quiesce took 5.8 seconds. Will release after: 10.599999999999998 2024-05-14T17:10:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 -- 172.21.15.99:0/1946388781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 msgr2=0x7fb208102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/1946388781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208102ed0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009930 tx=0x7fb1fc02a420 comp rx=0 tx=0).stop 2024-05-14T17:10:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 -- 172.21.15.99:0/1946388781 shutdown_connections 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/1946388781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb208103410 0x7fb20810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/1946388781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208102ed0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/1946388781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb208108a60 0x7fb208108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.391+0000 7fb20d1d0640 1 -- 172.21.15.99:0/1946388781 >> 172.21.15.99:0/1946388781 conn(0x7fb2080fe520 msgr2=0x7fb208100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.392+0000 7fb20d1d0640 1 -- 172.21.15.99:0/1946388781 shutdown_connections 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.392+0000 7fb20d1d0640 1 -- 172.21.15.99:0/1946388781 wait complete. 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.392+0000 7fb20d1d0640 1 Processor -- start 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.392+0000 7fb20d1d0640 1 -- start start 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb208103410 0x7fb208075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb208108a60 0x7fb2080757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112d20 con 0x7fb208102a50 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112ba0 con 0x7fb208108a60 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112ea0 con 0x7fb208103410 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56962/0 (socket says 172.21.15.99:56962) 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 -- 172.21.15.99:0/3591976562 learned_addr learned my addr 172.21.15.99:0/3591976562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20659c640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb208103410 0x7fb208075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 -- 172.21.15.99:0/3591976562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb208103410 msgr2=0x7fb208075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb208103410 0x7fb208075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 -- 172.21.15.99:0/3591976562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb208108a60 msgr2=0x7fb2080757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb208108a60 0x7fb2080757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 -- 172.21.15.99:0/3591976562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2081a7740 con 0x7fb208102a50 2024-05-14T17:10:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208074d50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800d770 tx=0x7fb1f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb1effff640 1 -- 172.21.15.99:0/3591976562 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8017070 con 0x7fb208102a50 2024-05-14T17:10:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb1effff640 1 -- 172.21.15.99:0/3591976562 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f80054d0 con 0x7fb208102a50 2024-05-14T17:10:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081a7a30 con 0x7fb208102a50 2024-05-14T17:10:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb1effff640 1 -- 172.21.15.99:0/3591976562 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004c30 con 0x7fb208102a50 2024-05-14T17:10:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.393+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a7f00 con 0x7fb208102a50 2024-05-14T17:10:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb1effff640 1 -- 172.21.15.99:0/3591976562 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb1f8004df0 con 0x7fb208102a50 2024-05-14T17:10:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 Processor -- start 2024-05-14T17:10:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 -- start start 2024-05-14T17:10:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45718cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45723590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45725ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55db4568bc30 con 0x55db45723ad0 2024-05-14T17:10:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55db4567b7d0 con 0x55db45719210 2024-05-14T17:10:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.396+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55db4567f810 con 0x55db457188f0 2024-05-14T17:10:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45725ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45725ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56970/0 (socket says 172.21.15.99:56970) 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 -- 172.21.15.99:0/1227944744 learned_addr learned my addr 172.21.15.99:0/1227944744 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb206d9d640 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45723590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20759e640 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45718cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb1effff640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb1f00b2a10 0x7fb1f00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20759e640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb1f00b2a10 0x7fb1f00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 -- 172.21.15.99:0/1227944744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 msgr2=0x55db45718cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45718cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 -- 172.21.15.99:0/1227944744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 msgr2=0x55db45723590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45723590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 -- 172.21.15.99:0/1227944744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db45390210 con 0x55db45723ad0 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb1effff640 1 -- 172.21.15.99:0/3591976562 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb1f80e2130 con 0x7fb208102a50 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20659c640 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45725ec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009930 tx=0x7fb1fc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b29ca451c98213c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.407 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb20759e640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb1f00b2a10 0x7fb1f00b4ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb200007ce0 tx=0x7fb200004a40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb1ed7fa640 1 -- 172.21.15.99:0/1227944744 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03f070 con 0x55db45723ad0 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb1ed7fa640 1 -- 172.21.15.99:0/1227944744 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1fc0316a0 con 0x55db45723ad0 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.397+0000 7fb1ed7fa640 1 -- 172.21.15.99:0/1227944744 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc035070 con 0x55db45723ad0 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1227944744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 msgr2=0x55db45725ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45725ec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009930 tx=0x7fb1fc032ce0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1227944744 shutdown_connections 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45725ec0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45723590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1227944744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45718cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1227944744 >> 172.21.15.99:0/1227944744 conn(0x55db45714e00 msgr2=0x55db457151e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1227944744 shutdown_connections 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1227944744 wait complete. 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 -- start start 2024-05-14T17:10:50.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.398+0000 7fb20ffc5740 1 Processor -- start 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45723410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45720700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45720c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55db45361790 con 0x55db45723ad0 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55db45637890 con 0x55db45719210 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55db45726740 con 0x55db457188f0 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 10 client.0 _ll_register_callbacks cb 0x55db4563ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45720c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45720c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56976/0 (socket says 172.21.15.99:56976) 2024-05-14T17:10:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 -- 172.21.15.99:0/3470675141 learned_addr learned my addr 172.21.15.99:0/3470675141 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45723410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20659c640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45720700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 -- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 msgr2=0x55db45723410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45723410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 -- 172.21.15.99:0/3470675141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 msgr2=0x55db45720700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45720700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 -- 172.21.15.99:0/3470675141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4545c620 con 0x55db45723ad0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20759e640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45720c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb2000119f0 tx=0x7fb200005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51364eb4c3fab939 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb1eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb1eaff5640 1 -- 172.21.15.99:0/3470675141 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200007e50 con 0x55db45723ad0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb1eaff5640 1 -- 172.21.15.99:0/3470675141 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb200010b60 con 0x55db45723ad0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1fc0316a0 con 0x55db45723ad0 2024-05-14T17:10:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.399+0000 7fb1eaff5640 1 -- 172.21.15.99:0/3470675141 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb20002c050 con 0x55db45723ad0 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db457321e0 con 0x55db45723ad0 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb1eaff5640 1 -- 172.21.15.99:0/3470675141 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1fc0316a0 con 0x55db45723ad0 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb1eaff5640 1 -- 172.21.15.99:0/3470675141 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb20000f040 con 0x55db45723ad0 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 20 client.16152 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db45733800 0x55db45735ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 4 client.16152 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db4568db80 con 0x55db45733800 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.400+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db45733800 0x55db45735ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db45733800 0x55db45735ca0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a73928e4969b2aa server_cookie=af7b8ce472d268be in_seq=0 out_seq=0 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb1eaff5640 10 client.16152 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb1eaff5640 1 -- 172.21.15.99:0/3470675141 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db4568db80 con 0x55db45733800 2024-05-14T17:10:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb1eaff5640 10 client.16152 handle_command_reply: tid=0 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 client.16152 shutdown 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 msgr2=0x55db45720c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45720c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb2000119f0 tx=0x7fb200005ec0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 shutdown_connections 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db45733800 0x55db45735ca0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45723ad0 0x55db45720c40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb1eaff5640 0 client.16152 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45720700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3470675141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45723410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 >> 172.21.15.99:0/3470675141 conn(0x55db45714e00 msgr2=0x55db45714100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.401+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 shutdown_connections 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.402+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3470675141 wait complete. 2024-05-14T17:10:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.402+0000 7fb20ffc5740 20 client.16152 trim_cache size 0 max 16384 2024-05-14T17:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 Processor -- start 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 -- start start 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 0x55db45722f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45713f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db45723ad0 0x55db4573b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081b3120 con 0x55db457188f0 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081b0990 con 0x55db45719210 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081ae800 con 0x55db45723ad0 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 0x55db45722f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 0x55db45722f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56992/0 (socket says 172.21.15.99:56992) 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.416+0000 7fb206d9d640 1 -- 172.21.15.99:0/3422110017 learned_addr learned my addr 172.21.15.99:0/3422110017 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb206d9d640 1 -- 172.21.15.99:0/3422110017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db45723ad0 msgr2=0x55db4573b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db45723ad0 0x55db4573b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb206d9d640 1 -- 172.21.15.99:0/3422110017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 msgr2=0x55db45713f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45713f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb206d9d640 1 -- 172.21.15.99:0/3422110017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db45714c30 con 0x55db457188f0 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb206d9d640 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 0x55db45722f40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80a90a0 tx=0x7fb1f8091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ab62cdcf1b53eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb1ecff9640 1 -- 172.21.15.99:0/3422110017 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8020040 con 0x55db457188f0 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb1ecff9640 1 -- 172.21.15.99:0/3422110017 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f80ad7a0 con 0x55db457188f0 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb1ecff9640 1 -- 172.21.15.99:0/3422110017 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80b1dc0 con 0x55db457188f0 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3422110017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 msgr2=0x55db45722f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 0x55db45722f40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80a90a0 tx=0x7fb1f8091040 comp rx=0 tx=0).stop 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3422110017 shutdown_connections 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db45723ad0 0x55db4573b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45719210 0x55db45713f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/3422110017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db457188f0 0x55db45722f40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.417+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3422110017 >> 172.21.15.99:0/3422110017 conn(0x55db45714e00 msgr2=0x55db45713870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3422110017 shutdown_connections 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 -- 172.21.15.99:0/3422110017 wait complete. 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 -- start start 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 Processor -- start 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45727640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 0x55db45727b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45723ad0 0x55db457280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081aee50 con 0x55db45719210 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081b02c0 con 0x55db45723ad0 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081af520 con 0x55db457188f0 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20ffc5740 10 client.0 _ll_register_callbacks cb 0x7fb200007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 0x55db45727b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 0x55db45727b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56996/0 (socket says 172.21.15.99:56996) 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 -- 172.21.15.99:0/1487987941 learned_addr learned my addr 172.21.15.99:0/1487987941 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:10:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20759e640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45723ad0 0x55db457280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb206d9d640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45727640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 -- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 msgr2=0x55db45727640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45727640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 -- 172.21.15.99:0/1487987941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45723ad0 msgr2=0x55db457280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45723ad0 0x55db457280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb20659c640 1 -- 172.21.15.99:0/1487987941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4579d260 con 0x55db45719210 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.418+0000 7fb206d9d640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45727640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20659c640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 0x55db45727b80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009930 tx=0x7fb1fc005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59135db3288d525b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb1eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:10:50.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb1eaff5640 1 -- 172.21.15.99:0/1487987941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc047070 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb1eaff5640 1 -- 172.21.15.99:0/1487987941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fb1fc004450 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f80ad7a0 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb1eaff5640 1 -- 172.21.15.99:0/1487987941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1fc053050 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 10 client.16158 fetch_fsmap learned FSMap version 29 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db4579e570 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb1eaff5640 1 -- 172.21.15.99:0/1487987941 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f80ad7a0 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb1eaff5640 1 -- 172.21.15.99:0/1487987941 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb1fc004c10 con 0x55db45719210 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 10 client.16158 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 20 client.16158 populate_metadata read hostname 'smithi099' 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db4579e8c0 0x55db457a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 4 client.16158 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.419+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db45687360 con 0x55db4579e8c0 2024-05-14T17:10:50.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.420+0000 7fb206d9d640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db4579e8c0 0x55db457a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.420+0000 7fb1eaff5640 10 client.16158 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.420+0000 7fb206d9d640 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db4579e8c0 0x55db457a0cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81103db81e494f1e server_cookie=b132a059a0b2e952 in_seq=0 out_seq=0 2024-05-14T17:10:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb1eaff5640 1 -- 172.21.15.99:0/1487987941 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55db45687360 con 0x55db4579e8c0 2024-05-14T17:10:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb1eaff5640 10 client.16158 handle_command_reply: tid=0 2024-05-14T17:10:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 client.16158 shutdown 2024-05-14T17:10:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 msgr2=0x55db45727b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 0x55db45727b80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009930 tx=0x7fb1fc005a40 comp rx=0 tx=0).stop 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 shutdown_connections 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55db4579e8c0 0x55db457a0cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db45723ad0 0x55db457280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db45719210 0x55db45727b80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb1eaff5640 0 client.16158 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 --2- 172.21.15.99:0/1487987941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db457188f0 0x55db45727640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 >> 172.21.15.99:0/1487987941 conn(0x55db45714e00 msgr2=0x55db45714100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 shutdown_connections 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 1 -- 172.21.15.99:0/1487987941 wait complete. 2024-05-14T17:10:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.453+0000 7fb20ffc5740 20 client.16158 trim_cache size 0 max 16384 2024-05-14T17:10:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:10:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:10:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb1f00b2a10 msgr2=0x7fb1f00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb1f00b2a10 0x7fb1f00b4ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb200007ce0 tx=0x7fb200004a40 comp rx=0 tx=0).stop 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 msgr2=0x7fb208074d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208074d50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800d770 tx=0x7fb1f800dc30 comp rx=0 tx=0).stop 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 shutdown_connections 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb1f00b2a10 0x7fb1f00b4ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb208108a60 0x7fb2080757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb208103410 0x7fb208075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 --2- 172.21.15.99:0/3591976562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb208102a50 0x7fb208074d50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 >> 172.21.15.99:0/3591976562 conn(0x7fb2080fe520 msgr2=0x7fb2080fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 shutdown_connections 2024-05-14T17:10:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:10:50.454+0000 7fb20d1d0640 1 -- 172.21.15.99:0/3591976562 wait complete. 2024-05-14T17:10:50.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.44 MB/sec execute 182 sec latency 1257.337 ms 2024-05-14T17:10:51.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:50 smithi150 ceph-mon[36204]: pgmap v221: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 928 KiB/s wr, 131 op/s 2024-05-14T17:10:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:50 smithi099 ceph-mon[31851]: pgmap v221: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 928 KiB/s wr, 131 op/s 2024-05-14T17:10:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:50 smithi193 ceph-mon[35962]: pgmap v221: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 928 KiB/s wr, 131 op/s 2024-05-14T17:10:51.534 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.40 MB/sec execute 183 sec latency 2257.430 ms 2024-05-14T17:10:52.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.37 MB/sec execute 184 sec latency 3257.471 ms 2024-05-14T17:10:53.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:52 smithi150 ceph-mon[36204]: pgmap v222: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 601 KiB/s wr, 76 op/s 2024-05-14T17:10:53.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:52 smithi099 ceph-mon[31851]: pgmap v222: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 601 KiB/s wr, 76 op/s 2024-05-14T17:10:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:52 smithi193 ceph-mon[35962]: pgmap v222: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 601 KiB/s wr, 76 op/s 2024-05-14T17:10:53.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.33 MB/sec execute 185 sec latency 4257.596 ms 2024-05-14T17:10:54.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:53 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:54.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:53 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:54.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:53 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:54.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.30 MB/sec execute 186 sec latency 5257.730 ms 2024-05-14T17:10:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:54 smithi150 ceph-mon[36204]: pgmap v223: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 918 KiB/s wr, 97 op/s 2024-05-14T17:10:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:54 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:54 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:55.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:54 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:10:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:54 smithi099 ceph-mon[31851]: pgmap v223: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 918 KiB/s wr, 97 op/s 2024-05-14T17:10:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:54 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:54 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:54 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:10:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:54 smithi193 ceph-mon[35962]: pgmap v223: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 918 KiB/s wr, 97 op/s 2024-05-14T17:10:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:54 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:54 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:54 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:10:55.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.26 MB/sec execute 187 sec latency 6257.772 ms 2024-05-14T17:10:56.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.23 MB/sec execute 188 sec latency 7257.930 ms 2024-05-14T17:10:57.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:56 smithi150 ceph-mon[36204]: pgmap v224: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 636 KiB/s wr, 59 op/s 2024-05-14T17:10:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:56 smithi099 ceph-mon[31851]: pgmap v224: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 636 KiB/s wr, 59 op/s 2024-05-14T17:10:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:56 smithi193 ceph-mon[35962]: pgmap v224: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 636 KiB/s wr, 59 op/s 2024-05-14T17:10:57.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.20 MB/sec execute 189 sec latency 8258.030 ms 2024-05-14T17:10:58.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.16 MB/sec execute 190 sec latency 9258.092 ms 2024-05-14T17:10:59.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:10:58 smithi150 ceph-mon[36204]: pgmap v225: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 636 KiB/s wr, 59 op/s 2024-05-14T17:10:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:10:58 smithi099 ceph-mon[31851]: pgmap v225: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 636 KiB/s wr, 59 op/s 2024-05-14T17:10:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:10:58 smithi193 ceph-mon[35962]: pgmap v225: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 636 KiB/s wr, 59 op/s 2024-05-14T17:10:59.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.13 MB/sec execute 191 sec latency 10258.186 ms 2024-05-14T17:11:00.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.10 MB/sec execute 192 sec latency 11258.222 ms 2024-05-14T17:11:00.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:11:00.858 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:01.183 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:00 smithi099 ceph-mon[31851]: pgmap v226: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 892 KiB/s wr, 72 op/s 2024-05-14T17:11:01.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:00 smithi150 ceph-mon[36204]: pgmap v226: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 892 KiB/s wr, 72 op/s 2024-05-14T17:11:01.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:00 smithi193 ceph-mon[35962]: pgmap v226: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 892 KiB/s wr, 72 op/s 2024-05-14T17:11:01.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.07 MB/sec execute 193 sec latency 12258.261 ms 2024-05-14T17:11:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 -- 172.21.15.99:0/1794493674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c108620 msgr2=0x7f4d0c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 --2- 172.21.15.99:0/1794493674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c108620 0x7f4d0c108a00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b0a0 tx=0x7f4d0400b3b0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 -- 172.21.15.99:0/1794493674 shutdown_connections 2024-05-14T17:11:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 --2- 172.21.15.99:0/1794493674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c1028c0 0x7f4d0c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 --2- 172.21.15.99:0/1794493674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c108620 0x7f4d0c108a00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 --2- 172.21.15.99:0/1794493674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c101f00 0x7f4d0c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 -- 172.21.15.99:0/1794493674 >> 172.21.15.99:0/1794493674 conn(0x7f4d0c077780 msgr2=0x7f4d0c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 -- 172.21.15.99:0/1794493674 shutdown_connections 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.458+0000 7f4d1090e700 1 -- 172.21.15.99:0/1794493674 wait complete. 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.459+0000 7f4d1090e700 1 Processor -- start 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.459+0000 7f4d1090e700 1 -- start start 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 0x7f4d0c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c1028c0 0x7f4d0c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c108620 0x7f4d0c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c113810 con 0x7f4d0c101f00 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c113690 con 0x7f4d0c1028c0 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c113990 con 0x7f4d0c108620 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 0x7f4d0c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 0x7f4d0c199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51864/0 (socket says 172.21.15.99:51864) 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 -- 172.21.15.99:0/447768118 learned_addr learned my addr 172.21.15.99:0/447768118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c1028c0 0x7f4d0c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 -- 172.21.15.99:0/447768118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c108620 msgr2=0x7f4d0c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0ad9d700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c108620 0x7f4d0c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c108620 0x7f4d0c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 -- 172.21.15.99:0/447768118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c1028c0 msgr2=0x7f4d0c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c1028c0 0x7f4d0c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 -- 172.21.15.99:0/447768118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c19ed20 con 0x7f4d0c101f00 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c1028c0 0x7f4d0c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0ad9d700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c108620 0x7f4d0c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d0a59c700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 0x7f4d0c199d60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b070 tx=0x7f4d04004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04021070 con 0x7f4d0c101f00 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d04007da0 con 0x7f4d0c101f00 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c19efb0 con 0x7f4d0c101f00 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d040053a0 con 0x7f4d0c101f00 2024-05-14T17:11:02.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.460+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c1a6890 con 0x7f4d0c101f00 2024-05-14T17:11:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.461+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0c06a230 con 0x7f4d0c101f00 2024-05-14T17:11:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.463+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4d04024070 con 0x7f4d0c101f00 2024-05-14T17:11:02.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.463+0000 7f4d037fe700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4cf80b2a70 0x7f4cf80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.464+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4cf80b2a70 0x7f4cf80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.464+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f4d0400f020 con 0x7f4d0c101f00 2024-05-14T17:11:02.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.464+0000 7f4d09d9b700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4cf80b2a70 0x7f4cf80b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c077df0 tx=0x7f4cf4008040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:02.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.467+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d0402b020 con 0x7f4d0c101f00 2024-05-14T17:11:02.535 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.04 MB/sec execute 194 sec latency 13258.324 ms 2024-05-14T17:11:02.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.722+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d0c19b070 con 0x7f4d0c101f00 2024-05-14T17:11:02.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.723+0000 7f4d037fe700 1 -- 172.21.15.99:0/447768118 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11879 (secure 0 0 0) 0x7f4d040abb00 con 0x7f4d0c101f00 2024-05-14T17:11:02.724 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:11:02.724 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:10:04.586724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/3113222034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3113222034},{"type":"v1","addr":"172.21.15.193:6835","nonce":3113222034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:11:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4cf80b2a70 msgr2=0x7f4cf80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4cf80b2a70 0x7f4cf80b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c077df0 tx=0x7f4cf4008040 comp rx=0 tx=0).stop 2024-05-14T17:11:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 msgr2=0x7f4d0c199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 0x7f4d0c199d60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400b070 tx=0x7f4d04004f10 comp rx=0 tx=0).stop 2024-05-14T17:11:02.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 shutdown_connections 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4cf80b2a70 0x7f4cf80b4f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d0c108620 0x7f4d0c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c101f00 0x7f4d0c199d60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 --2- 172.21.15.99:0/447768118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0c1028c0 0x7f4d0c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.725+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 >> 172.21.15.99:0/447768118 conn(0x7f4d0c077780 msgr2=0x7f4d0c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.726+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 shutdown_connections 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:02.726+0000 7f4d1090e700 1 -- 172.21.15.99:0/447768118 wait complete. 2024-05-14T17:11:02.727 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-05-14T17:11:03.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:02 smithi099 ceph-mon[31851]: pgmap v227: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 573 KiB/s wr, 34 op/s 2024-05-14T17:11:03.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:02 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/447768118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:03.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:02 smithi150 ceph-mon[36204]: pgmap v227: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 573 KiB/s wr, 34 op/s 2024-05-14T17:11:03.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:02 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/447768118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:02 smithi193 ceph-mon[35962]: pgmap v227: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 573 KiB/s wr, 34 op/s 2024-05-14T17:11:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:02 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/447768118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:03.338 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 379b651a --release --await' 2024-05-14T17:11:03.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 379b651a --release --await 2024-05-14T17:11:03.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 6.01 MB/sec execute 195 sec latency 14258.353 ms 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 -- 172.21.15.99:0/2075409974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394107940 msgr2=0x7fa394107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 --2- 172.21.15.99:0/2075409974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394107940 0x7fa394107d20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa384009930 tx=0x7fa38402a420 comp rx=0 tx=0).stop 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 -- 172.21.15.99:0/2075409974 shutdown_connections 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 --2- 172.21.15.99:0/2075409974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394102340 0x7fa39410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 --2- 172.21.15.99:0/2075409974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa394101980 0x7fa394101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 --2- 172.21.15.99:0/2075409974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394107940 0x7fa394107d20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.545+0000 7fa399312640 1 -- 172.21.15.99:0/2075409974 >> 172.21.15.99:0/2075409974 conn(0x7fa394078ad0 msgr2=0x7fa394078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.546+0000 7fa399312640 1 -- 172.21.15.99:0/2075409974 shutdown_connections 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.546+0000 7fa399312640 1 -- 172.21.15.99:0/2075409974 wait complete. 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.546+0000 7fa399312640 1 Processor -- start 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.546+0000 7fa399312640 1 -- start start 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa399312640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 0x7fa394199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa399312640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa394102340 0x7fa39419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa399312640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394107940 0x7fa3941a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa399312640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa394111bd0 con 0x7fa394101980 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa399312640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa394111a50 con 0x7fa394102340 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa399312640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa394111d50 con 0x7fa394107940 2024-05-14T17:11:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 0x7fa394199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 0x7fa394199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51880/0 (socket says 172.21.15.99:51880) 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 -- 172.21.15.99:0/760495670 learned_addr learned my addr 172.21.15.99:0/760495670 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa3937fe640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394107940 0x7fa3941a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa3927fc640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa394102340 0x7fa39419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 -- 172.21.15.99:0/760495670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394107940 msgr2=0x7fa3941a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394107940 0x7fa3941a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 -- 172.21.15.99:0/760495670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa394102340 msgr2=0x7fa39419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa394102340 0x7fa39419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 -- 172.21.15.99:0/760495670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a55b0 con 0x7fa394101980 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa392ffd640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 0x7fa394199ff0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa384004770 tx=0x7fa3840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.547+0000 7fa37bfff640 1 -- 172.21.15.99:0/760495670 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38403f070 con 0x7fa394101980 2024-05-14T17:11:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.548+0000 7fa37bfff640 1 -- 172.21.15.99:0/760495670 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38403b460 con 0x7fa394101980 2024-05-14T17:11:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.548+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a5780 con 0x7fa394101980 2024-05-14T17:11:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.548+0000 7fa37bfff640 1 -- 172.21.15.99:0/760495670 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384035070 con 0x7fa394101980 2024-05-14T17:11:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.548+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a5b80 con 0x7fa394101980 2024-05-14T17:11:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.550+0000 7fa39c01f740 1 Processor -- start 2024-05-14T17:11:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.550+0000 7fa39c01f740 1 -- start start 2024-05-14T17:11:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a1d990 0x562465a1dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19400 0x562465a19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa39c01f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562465982c30 con 0x562465a19dc0 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa39c01f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5624659727d0 con 0x562465a1d990 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa39c01f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562465976810 con 0x562465a19400 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa37bfff640 1 -- 172.21.15.99:0/760495670 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa38403b8d0 con 0x7fa394101980 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a258e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51892/0 (socket says 172.21.15.99:51892) 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 -- 172.21.15.99:0/2408061705 learned_addr learned my addr 172.21.15.99:0/2408061705 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3937fe640 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a1d990 0x562465a1dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa392ffd640 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19400 0x562465a19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 -- 172.21.15.99:0/2408061705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19400 msgr2=0x562465a19880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19400 0x562465a19880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 -- 172.21.15.99:0/2408061705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a1d990 msgr2=0x562465a1dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a1d990 0x562465a1dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 -- 172.21.15.99:0/2408061705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562465687210 con 0x562465a19dc0 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.551+0000 7fa3927fc640 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a258e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa38800bc80 tx=0x7fa38800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4721f40befb3ff08 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.552+0000 7fa37bfff640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa37c0b2a10 0x7fa37c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.552+0000 7fa3797fa640 1 -- 172.21.15.99:0/2408061705 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388015070 con 0x562465a19dc0 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.552+0000 7fa3797fa640 1 -- 172.21.15.99:0/2408061705 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38801a910 con 0x562465a19dc0 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.552+0000 7fa3797fa640 1 -- 172.21.15.99:0/2408061705 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3880118b0 con 0x562465a19dc0 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.552+0000 7fa3937fe640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa37c0b2a10 0x7fa37c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.552+0000 7fa37bfff640 1 -- 172.21.15.99:0/760495670 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fa384102a40 con 0x7fa394101980 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 -- 172.21.15.99:0/2408061705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 msgr2=0x562465a258e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a258e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa38800bc80 tx=0x7fa38800f730 comp rx=0 tx=0).stop 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa3937fe640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa37c0b2a10 0x7fa37c0b4ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007e50 tx=0x7fa38c007600 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 -- 172.21.15.99:0/2408061705 shutdown_connections 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a258e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19400 0x562465a19880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 --2- 172.21.15.99:0/2408061705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a1d990 0x562465a1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 -- 172.21.15.99:0/2408061705 >> 172.21.15.99:0/2408061705 conn(0x562465a0b8b0 msgr2=0x562465a0bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 -- 172.21.15.99:0/2408061705 shutdown_connections 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 -- 172.21.15.99:0/2408061705 wait complete. 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 -- start start 2024-05-14T17:11:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.553+0000 7fa39c01f740 1 Processor -- start 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a22990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a1d990 0x562465a22ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562465658790 con 0x562465a19dc0 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56246592e890 con 0x562465a19400 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562465a25f80 con 0x562465a1d990 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa39c01f740 10 client.0 _ll_register_callbacks cb 0x562465931d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a22990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a22990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51894/0 (socket says 172.21.15.99:51894) 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 -- 172.21.15.99:0/4067945163 learned_addr learned my addr 172.21.15.99:0/4067945163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa392ffd640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3937fe640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a1d990 0x562465a22ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 -- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a1d990 msgr2=0x562465a22ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a1d990 0x562465a22ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 -- 172.21.15.99:0/4067945163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 msgr2=0x562465a256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 -- 172.21.15.99:0/4067945163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562465753620 con 0x562465a19dc0 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3937fe640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a1d990 0x562465a22ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa3927fc640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a22990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa388005cd0 tx=0x7fa38800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7b56f9a52c30ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.554+0000 7fa376ff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa376ff5640 1 -- 172.21.15.99:0/4067945163 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388015050 con 0x562465a19dc0 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa38801a910 con 0x562465a19dc0 2024-05-14T17:11:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa376ff5640 1 -- 172.21.15.99:0/4067945163 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fa38801aac0 con 0x562465a19dc0 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa376ff5640 1 -- 172.21.15.99:0/4067945163 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa388031050 con 0x562465a19dc0 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 10 client.16176 fetch_fsmap learned FSMap version 29 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562465a31960 con 0x562465a19dc0 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa376ff5640 1 -- 172.21.15.99:0/4067945163 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38801a910 con 0x562465a19dc0 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa376ff5640 1 -- 172.21.15.99:0/4067945163 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa38802c030 con 0x562465a19dc0 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 10 client.16176 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 20 client.16176 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a33180 0x562465a35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 4 client.16176 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.555+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562465984b80 con 0x562465a33180 2024-05-14T17:11:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.556+0000 7fa392ffd640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a33180 0x562465a35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.556+0000 7fa392ffd640 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a33180 0x562465a35600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b58877204e6ad84b server_cookie=30637f30e6d98c19 in_seq=0 out_seq=0 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.556+0000 7fa376ff5640 10 client.16176 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa376ff5640 1 -- 172.21.15.99:0/4067945163 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562465984b80 con 0x562465a33180 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa376ff5640 10 client.16176 handle_command_reply: tid=0 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 client.16176 shutdown 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 msgr2=0x562465a22990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a22990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa388005cd0 tx=0x7fa38800bfa0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 shutdown_connections 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a33180 0x562465a35600 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a1d990 0x562465a22ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a19dc0 0x562465a22990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 --2- 172.21.15.99:0/4067945163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 >> 172.21.15.99:0/4067945163 conn(0x562465a0b8b0 msgr2=0x562465a18c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa376ff5640 0 client.16176 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 shutdown_connections 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 1 -- 172.21.15.99:0/4067945163 wait complete. 2024-05-14T17:11:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.557+0000 7fa39c01f740 20 client.16176 trim_cache size 0 max 16384 2024-05-14T17:11:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 Processor -- start 2024-05-14T17:11:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 -- start start 2024-05-14T17:11:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a3af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941aecf0 con 0x562465a1d990 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ac560 con 0x562465a19400 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa39c01f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941aa3d0 con 0x562465a19dc0 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a3af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a3af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51910/0 (socket says 172.21.15.99:51910) 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 -- 172.21.15.99:0/3206360799 learned_addr learned my addr 172.21.15.99:0/3206360799 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3927fc640 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 -- 172.21.15.99:0/3206360799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 msgr2=0x562465a251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa392ffd640 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 -- 172.21.15.99:0/3206360799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 msgr2=0x562465a2eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a2eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3937fe640 1 -- 172.21.15.99:0/3206360799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562465a06510 con 0x562465a1d990 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.577+0000 7fa3927fc640 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.578+0000 7fa3937fe640 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a3af10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa38c01d4f0 tx=0x7fa38c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca743b898013f267 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.578+0000 7fa378ff9640 1 -- 172.21.15.99:0/3206360799 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c01b810 con 0x562465a1d990 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.578+0000 7fa378ff9640 1 -- 172.21.15.99:0/3206360799 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38c00e3b0 con 0x562465a1d990 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.578+0000 7fa378ff9640 1 -- 172.21.15.99:0/3206360799 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c01be60 con 0x562465a1d990 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- 172.21.15.99:0/3206360799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 msgr2=0x562465a3af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a3af10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa38c01d4f0 tx=0x7fa38c014000 comp rx=0 tx=0).stop 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- 172.21.15.99:0/3206360799 shutdown_connections 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a3af10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- 172.21.15.99:0/3206360799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a2eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- 172.21.15.99:0/3206360799 >> 172.21.15.99:0/3206360799 conn(0x562465a0b8b0 msgr2=0x562465a21260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- 172.21.15.99:0/3206360799 shutdown_connections 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- 172.21.15.99:0/3206360799 wait complete. 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- start start 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 Processor -- start 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a3b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a2b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a2b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941aaa20 con 0x562465a1d990 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941abe90 con 0x562465a19400 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ab0f0 con 0x562465a19dc0 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa39c01f740 10 client.0 _ll_register_callbacks cb 0x7fa38800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa3937fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a2b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa3937fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a2b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51922/0 (socket says 172.21.15.99:51922) 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa3937fe640 1 -- 172.21.15.99:0/623329958 learned_addr learned my addr 172.21.15.99:0/623329958 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.579+0000 7fa392ffd640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a3b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3927fc640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a2b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3937fe640 1 -- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 msgr2=0x562465a2b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3937fe640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a2b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3937fe640 1 -- 172.21.15.99:0/623329958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 msgr2=0x562465a3b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3937fe640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a3b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3937fe640 1 -- 172.21.15.99:0/623329958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562465a3a000 con 0x562465a1d990 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa3937fe640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a2b780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa38c01d4c0 tx=0x7fa38c004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6453f5a4f6fa1fda server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa376ff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa376ff5640 1 -- 172.21.15.99:0/623329958 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c033030 con 0x562465a1d990 2024-05-14T17:11:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa376ff5640 1 -- 172.21.15.99:0/623329958 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7fa38c02c040 con 0x562465a1d990 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa38c00e3b0 con 0x562465a1d990 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa376ff5640 1 -- 172.21.15.99:0/623329958 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa38c03d050 con 0x562465a1d990 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa39c01f740 10 client.16182 fetch_fsmap learned FSMap version 29 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562465a9ce20 con 0x562465a1d990 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.580+0000 7fa376ff5640 1 -- 172.21.15.99:0/623329958 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38c00e3b0 con 0x562465a1d990 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa376ff5640 1 -- 172.21.15.99:0/623329958 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa38c0053a0 con 0x562465a1d990 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa39c01f740 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa39c01f740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa39c01f740 20 client.16182 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa39c01f740 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a9d4f0 0x562465a9f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa39c01f740 4 client.16182 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56246597e360 con 0x562465a9d4f0 2024-05-14T17:11:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa392ffd640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a9d4f0 0x562465a9f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa392ffd640 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a9d4f0 0x562465a9f910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8acbc3c4644299c server_cookie=6f1a785c293f98cd in_seq=0 out_seq=0 2024-05-14T17:11:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:03.581+0000 7fa376ff5640 10 client.16182 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:03.600 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:04.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 356464 5.98 MB/sec execute 196 sec latency 15258.492 ms 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa376ff5640 1 -- 172.21.15.99:0/623329958 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56246597e360 con 0x562465a9d4f0 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa376ff5640 10 client.16182 handle_command_reply: tid=0 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 client.16182 shutdown 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 msgr2=0x562465a2b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a2b780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa38c01d4c0 tx=0x7fa38c004c70 comp rx=0 tx=0).stop 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 shutdown_connections 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562465a9d4f0 0x562465a9f910 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562465a1d990 0x562465a2b780 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa376ff5640 0 client.16182 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562465a19dc0 0x562465a2b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 --2- 172.21.15.99:0/623329958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562465a19400 0x562465a3b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 >> 172.21.15.99:0/623329958 conn(0x562465a0b8b0 msgr2=0x562465a20e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 shutdown_connections 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 1 -- 172.21.15.99:0/623329958 wait complete. 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.580+0000 7fa39c01f740 20 client.16182 trim_cache size 0 max 16384 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:11:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa37c0b2a10 msgr2=0x7fa37c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa37c0b2a10 0x7fa37c0b4ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007e50 tx=0x7fa38c007600 comp rx=0 tx=0).stop 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 msgr2=0x7fa394199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 0x7fa394199ff0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa384004770 tx=0x7fa3840042f0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 shutdown_connections 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa37c0b2a10 0x7fa37c0b4ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa394107940 0x7fa3941a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa394102340 0x7fa39419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 --2- 172.21.15.99:0/760495670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa394101980 0x7fa394199ff0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 >> 172.21.15.99:0/760495670 conn(0x7fa394078ad0 msgr2=0x7fa394104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 shutdown_connections 2024-05-14T17:11:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:04.581+0000 7fa399312640 1 -- 172.21.15.99:0/760495670 wait complete. 2024-05-14T17:11:05.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 -- 172.21.15.99:0/880719477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481037f0 msgr2=0x7f2a48103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 --2- 172.21.15.99:0/880719477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481037f0 0x7f2a48103bd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b0d0 tx=0x7f2a4400b3e0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 -- 172.21.15.99:0/880719477 shutdown_connections 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 --2- 172.21.15.99:0/880719477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a481088f0 0x7f2a48110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 --2- 172.21.15.99:0/880719477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481037f0 0x7f2a48103bd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 --2- 172.21.15.99:0/880719477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481041a0 0x7f2a481081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 -- 172.21.15.99:0/880719477 >> 172.21.15.99:0/880719477 conn(0x7f2a4807a6a0 msgr2=0x7f2a4807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 -- 172.21.15.99:0/880719477 shutdown_connections 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 -- 172.21.15.99:0/880719477 wait complete. 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.093+0000 7f2a505bf700 1 Processor -- start 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 -- start start 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a481037f0 0x7f2a4819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 0x7f2a4819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481088f0 0x7f2a48199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481136e0 con 0x7f2a481041a0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113560 con 0x7f2a481037f0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a505bf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113860 con 0x7f2a481088f0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 0x7f2a4819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 0x7f2a4819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55160/0 (socket says 172.21.15.99:55160) 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 -- 172.21.15.99:0/4112882574 learned_addr learned my addr 172.21.15.99:0/4112882574 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481088f0 0x7f2a48199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a481037f0 0x7f2a4819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 -- 172.21.15.99:0/4112882574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481088f0 msgr2=0x7f2a48199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481088f0 0x7f2a48199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 -- 172.21.15.99:0/4112882574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a481037f0 msgr2=0x7f2a4819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a481037f0 0x7f2a4819f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 -- 172.21.15.99:0/4112882574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4819a1a0 con 0x7f2a481041a0 2024-05-14T17:11:05.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481088f0 0x7f2a48199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.094+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 0x7f2a4819fd70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800d980 tx=0x7f2a3800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.095+0000 7f2a3f7fe700 1 -- 172.21.15.99:0/4112882574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38017070 con 0x7f2a481041a0 2024-05-14T17:11:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.095+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4819a490 con 0x7f2a481041a0 2024-05-14T17:11:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.095+0000 7f2a3f7fe700 1 -- 172.21.15.99:0/4112882574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38004830 con 0x7f2a481041a0 2024-05-14T17:11:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.095+0000 7f2a3f7fe700 1 -- 172.21.15.99:0/4112882574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38005440 con 0x7f2a481041a0 2024-05-14T17:11:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.095+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a6700 con 0x7f2a481041a0 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.097+0000 7f2a3f7fe700 1 -- 172.21.15.99:0/4112882574 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2a380055e0 con 0x7f2a481041a0 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.098+0000 7f2a3f7fe700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2a340b29f0 0x7f2a340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.098+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2a340b29f0 0x7f2a340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.098+0000 7f2a3f7fe700 1 -- 172.21.15.99:0/4112882574 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f2a380e2630 con 0x7f2a481041a0 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.098+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2a340b29f0 0x7f2a340b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400d010 tx=0x7f2a4400b040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 Processor -- start 2024-05-14T17:11:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 -- start start 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7ea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55964e743e70 con 0x55964e7e62c0 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55964e733a10 con 0x55964e7ea420 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55964e737a50 con 0x55964e7ead40 2024-05-14T17:11:05.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4eb5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4eb5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7e6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55172/0 (socket says 172.21.15.99:55172) 2024-05-14T17:11:05.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/3239574464 learned_addr learned my addr 172.21.15.99:0/3239574464 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:05.100 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:04 smithi099 ceph-mon[31851]: pgmap v228: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 887 KiB/s wr, 66 op/s 2024-05-14T17:11:05.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7ea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/3239574464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 msgr2=0x55964e7e5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.099+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7e5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/3239574464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 msgr2=0x55964e7ea800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7ea800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/3239574464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964e6dd0b0 con 0x55964e7e62c0 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7e6720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000bba0 tx=0x7f2a4000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bf948426627ee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a3cff9700 1 -- 172.21.15.99:0/3239574464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000dd00 con 0x55964e7e62c0 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a3cff9700 1 -- 172.21.15.99:0/3239574464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a400178d0 con 0x55964e7e62c0 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.100+0000 7f2a3cff9700 1 -- 172.21.15.99:0/3239574464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000f880 con 0x55964e7e62c0 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/3239574464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 msgr2=0x55964e7e6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7e6720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000bba0 tx=0x7f2a4000beb0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/3239574464 shutdown_connections 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7e6720 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7ea800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/3239574464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7e5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/3239574464 >> 172.21.15.99:0/3239574464 conn(0x55964e712eb0 msgr2=0x55964e7e04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/3239574464 shutdown_connections 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/3239574464 wait complete. 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 -- start start 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.101+0000 7f2a5ae55b80 1 Processor -- start 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 0x55964e7eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ead40 0x55964e7ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55964e63a0a0 con 0x55964e7ea420 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55964e69bea0 con 0x55964e7ead40 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55964e7f2880 con 0x55964e7e62c0 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 10 client.0 _ll_register_callbacks cb 0x55964e6cea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 0x55964e7eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 0x55964e7eeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55176/0 (socket says 172.21.15.99:55176) 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 -- 172.21.15.99:0/2816300770 learned_addr learned my addr 172.21.15.99:0/2816300770 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:05.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ead40 0x55964e7ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 -- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 msgr2=0x55964e7f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 -- 172.21.15.99:0/2816300770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ead40 msgr2=0x55964e7ef3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ead40 0x55964e7ef3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 -- 172.21.15.99:0/2816300770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964e6839d0 con 0x55964e7ea420 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ead40 0x55964e7ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 0x55964e7eeea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800d950 tx=0x7f2a380a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9358882ce548c6b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a31ffb700 1 -- 172.21.15.99:0/2816300770 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380adc60 con 0x55964e7ea420 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a31ffb700 1 -- 172.21.15.99:0/2816300770 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f2a380aabe0 con 0x55964e7ea420 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.102+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a400178d0 con 0x55964e7ea420 2024-05-14T17:11:05.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a31ffb700 1 -- 172.21.15.99:0/2816300770 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a380b2710 con 0x55964e7ea420 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 10 client.16194 fetch_fsmap learned FSMap version 29 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55964e6ed860 con 0x55964e7ea420 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a31ffb700 1 -- 172.21.15.99:0/2816300770 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a400178d0 con 0x55964e7ea420 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a31ffb700 1 -- 172.21.15.99:0/2816300770 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2a380b2280 con 0x55964e7ea420 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 10 client.16194 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 20 client.16194 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e7ff490 0x55964e801880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 4 client.16194 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55964e745dc0 con 0x55964e7ff490 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.103+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e7ff490 0x55964e801880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.104+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e7ff490 0x55964e801880 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c52ea4bbdea62d server_cookie=c5d04e3bc6426b18 in_seq=0 out_seq=0 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.104+0000 7f2a31ffb700 10 client.16194 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.104+0000 7f2a31ffb700 1 -- 172.21.15.99:0/2816300770 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55964e745dc0 con 0x55964e7ff490 2024-05-14T17:11:05.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.104+0000 7f2a31ffb700 10 client.16194 handle_command_reply: tid=0 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.104+0000 7f2a5ae55b80 1 client.16194 shutdown 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 msgr2=0x55964e7eeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 0x55964e7eeea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800d950 tx=0x7f2a380a5820 comp rx=0 tx=0).stop 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 shutdown_connections 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ead40 0x55964e7ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e7ff490 0x55964e801880 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/2816300770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ea420 0x55964e7eeea0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 >> 172.21.15.99:0/2816300770 conn(0x55964e712eb0 msgr2=0x55964e7dafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a31ffb700 0 client.16194 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 shutdown_connections 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/2816300770 wait complete. 2024-05-14T17:11:05.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.105+0000 7f2a5ae55b80 20 client.16194 trim_cache size 0 max 16384 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 Processor -- start 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 -- start start 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481b0780 con 0x55964e7e62c0 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481adfb0 con 0x55964e7ea420 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.125+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481abe20 con 0x55964e7ead40 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55182/0 (socket says 172.21.15.99:55182) 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 -- 172.21.15.99:0/863957643 learned_addr learned my addr 172.21.15.99:0/863957643 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:05.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 -- 172.21.15.99:0/863957643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 msgr2=0x55964e7f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 -- 172.21.15.99:0/863957643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 msgr2=0x55964e7db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7db0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 -- 172.21.15.99:0/863957643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964e7e6010 con 0x55964e7e62c0 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7ede80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b0d0 tx=0x7f2a4401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa7deb84d82a335 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a33fff700 1 -- 172.21.15.99:0/863957643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44007e10 con 0x55964e7e62c0 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a33fff700 1 -- 172.21.15.99:0/863957643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44039070 con 0x55964e7e62c0 2024-05-14T17:11:05.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.126+0000 7f2a33fff700 1 -- 172.21.15.99:0/863957643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4401c9d0 con 0x55964e7e62c0 2024-05-14T17:11:05.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/863957643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 msgr2=0x55964e7ede80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7ede80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b0d0 tx=0x7f2a4401f000 comp rx=0 tx=0).stop 2024-05-14T17:11:05.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/863957643 shutdown_connections 2024-05-14T17:11:05.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7ead40 0x55964e7f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7e62c0 0x55964e7ede80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/863957643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/863957643 >> 172.21.15.99:0/863957643 conn(0x55964e712eb0 msgr2=0x55964e7e47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/863957643 shutdown_connections 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/863957643 wait complete. 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 -- start start 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 Processor -- start 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.127+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 0x55964e7e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481ac470 con 0x55964e7ead40 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481ad8e0 con 0x55964e7ea420 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a481acb40 con 0x55964e7e62c0 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 10 client.0 _ll_register_callbacks cb 0x55964e6cea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 0x55964e7e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4e35b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 0x55964e7e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55198/0 (socket says 172.21.15.99:55198) 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/804376012 learned_addr learned my addr 172.21.15.99:0/804376012 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 msgr2=0x55964e7e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/804376012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 msgr2=0x55964e7e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 -- 172.21.15.99:0/804376012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964e7e9fe0 con 0x55964e7ead40 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4db5a700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a4eb5c700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 0x55964e7e4590 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2a40019040 tx=0x7f2a4000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89519d9a757d83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a31ffb700 1 -- 172.21.15.99:0/804376012 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40027020 con 0x55964e7ead40 2024-05-14T17:11:05.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a31ffb700 1 -- 172.21.15.99:0/804376012 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8521+0+0 (secure 0 0 0) 0x7f2a40005580 con 0x55964e7ead40 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a44039070 con 0x55964e7ead40 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.128+0000 7f2a31ffb700 1 -- 172.21.15.99:0/804376012 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a40004710 con 0x55964e7ead40 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 10 client.16200 fetch_fsmap learned FSMap version 29 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55964e86ee10 con 0x55964e7ead40 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a31ffb700 1 -- 172.21.15.99:0/804376012 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44039070 con 0x55964e7ead40 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a31ffb700 1 -- 172.21.15.99:0/804376012 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2a400239b0 con 0x55964e7ead40 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 10 client.16200 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 10 client.16200 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 20 client.16200 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e86f4b0 0x55964e8718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 4 client.16200 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "379b651a", "release": true, "await": true}] 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "379b651a", "release": true, "await": true}) -- 0x55964e73f5a0 con 0x55964e86f4b0 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.129+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e86f4b0 0x55964e8718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.130+0000 7f2a4e35b700 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e86f4b0 0x55964e8718d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e7ee701211a326 server_cookie=2173d729f297141b in_seq=0 out_seq=0 2024-05-14T17:11:05.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.130+0000 7f2a31ffb700 10 client.16200 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:05.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a31ffb700 1 -- 172.21.15.99:0/804376012 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55964e73f5a0 con 0x55964e86f4b0 2024-05-14T17:11:05.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a31ffb700 10 client.16200 handle_command_reply: tid=0 2024-05-14T17:11:05.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 client.16200 shutdown 2024-05-14T17:11:05.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 msgr2=0x55964e7e4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 0x55964e7e4590 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2a40019040 tx=0x7f2a4000b3e0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 shutdown_connections 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55964e86f4b0 0x55964e8718d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55964e7ead40 0x55964e7e4590 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55964e7e62c0 0x55964e7e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 --2- 172.21.15.99:0/804376012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55964e7ea420 0x55964e7e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 >> 172.21.15.99:0/804376012 conn(0x55964e712eb0 msgr2=0x55964e7f0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a31ffb700 0 client.16200 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 shutdown_connections 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.135+0000 7f2a5ae55b80 1 -- 172.21.15.99:0/804376012 wait complete. 2024-05-14T17:11:05.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a5ae55b80 20 client.16200 trim_cache size 0 max 16384 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 29, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 24, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "379b651a": { 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 24, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:11:05.137 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 136.4, 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:05.138 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2a340b29f0 msgr2=0x7f2a340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2a340b29f0 0x7f2a340b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400d010 tx=0x7f2a4400b040 comp rx=0 tx=0).stop 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 msgr2=0x7f2a4819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 0x7f2a4819fd70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800d980 tx=0x7f2a3800dd40 comp rx=0 tx=0).stop 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 shutdown_connections 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2a340b29f0 0x7f2a340b4eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a481088f0 0x7f2a48199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a481037f0 0x7f2a4819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 --2- 172.21.15.99:0/4112882574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a481041a0 0x7f2a4819fd70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 >> 172.21.15.99:0/4112882574 conn(0x7f2a4807a6a0 msgr2=0x7f2a48109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:05.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 shutdown_connections 2024-05-14T17:11:05.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:05.136+0000 7f2a505bf700 1 -- 172.21.15.99:0/4112882574 wait complete. 2024-05-14T17:11:05.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:04 smithi150 ceph-mon[36204]: pgmap v228: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 887 KiB/s wr, 66 op/s 2024-05-14T17:11:05.254 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:04 smithi193 ceph-mon[35962]: pgmap v228: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 887 KiB/s wr, 66 op/s 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: *** Caught signal (Segmentation fault) ** 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: in thread 7ff3fb0f0700 thread_name:md_log_replay 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff40c4b2d20] 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x559784701c1f] 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597847053b3] 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 4: (MDLog::_replay_thread()+0x617) [0x559784760197] 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559784769a11] 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff40c4a81ca] 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 7: clone() 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:04.886+0000 7ff3fb0f0700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: in thread 7ff3fb0f0700 thread_name:md_log_replay 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:05.255 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff40c4b2d20] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x559784701c1f] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597847053b3] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 4: (MDLog::_replay_thread()+0x617) [0x559784760197] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559784769a11] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff40c4a81ca] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 7: clone() 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 0> 2024-05-14T17:11:04.886+0000 7ff3fb0f0700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: in thread 7ff3fb0f0700 thread_name:md_log_replay 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff40c4b2d20] 2024-05-14T17:11:05.256 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x559784701c1f] 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597847053b3] 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 4: (MDLog::_replay_thread()+0x617) [0x559784760197] 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559784769a11] 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff40c4a81ca] 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 7: clone() 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 0> 2024-05-14T17:11:04.886+0000 7ff3fb0f0700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: in thread 7ff3fb0f0700 thread_name:md_log_replay 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:11:05.257 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff40c4b2d20] 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2: /usr/bin/ceph-mds(+0x54ac1f) [0x559784701c1f] 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5597847053b3] 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 4: (MDLog::_replay_thread()+0x617) [0x559784760197] 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559784769a11] 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff40c4a81ca] 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 7: clone() 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[67069]: 2024-05-14T17:11:05.258 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:04 smithi193 podman[72663]: 2024-05-14 17:11:04.955888244 +0000 UTC m=+0.022376295 container died ed371fb0d113495117ef712b0cd532b3ad1132afa68c3bad9b77daf0afcd5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514) 2024-05-14T17:11:05.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 357419 5.97 MB/sec execute 197 sec latency 15858.289 ms 2024-05-14T17:11:05.884 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '379b651a', total seconds elapsed: 21.7 2024-05-14T17:11:05.928 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:05 smithi193 podman[72663]: 2024-05-14 17:11:05.666165919 +0000 UTC m=+0.732653976 container remove ed371fb0d113495117ef712b0cd532b3ad1132afa68c3bad9b77daf0afcd5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-14T17:11:05.928 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:05 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-05-14T17:11:06.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:05 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-14T17:11:06.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:05 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service: Consumed 9.101s CPU time. 2024-05-14T17:11:06.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 358899 5.99 MB/sec execute 198 sec latency 24.206 ms 2024-05-14T17:11:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:06 smithi150 ceph-mon[36204]: pgmap v229: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 571 KiB/s wr, 44 op/s 2024-05-14T17:11:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:06 smithi150 ceph-mon[36204]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T17:11:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:06 smithi099 ceph-mon[31851]: pgmap v229: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 571 KiB/s wr, 44 op/s 2024-05-14T17:11:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:06 smithi099 ceph-mon[31851]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T17:11:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:06 smithi193 ceph-mon[35962]: pgmap v229: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 571 KiB/s wr, 44 op/s 2024-05-14T17:11:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:06 smithi193 ceph-mon[35962]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T17:11:07.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 359941 5.98 MB/sec execute 199 sec latency 380.672 ms 2024-05-14T17:11:08.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 359941 5.95 MB/sec execute 200 sec latency 1380.706 ms 2024-05-14T17:11:09.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:08 smithi150 ceph-mon[36204]: pgmap v231: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 685 KiB/s wr, 53 op/s 2024-05-14T17:11:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:08 smithi099 ceph-mon[31851]: pgmap v231: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 685 KiB/s wr, 53 op/s 2024-05-14T17:11:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:08 smithi193 ceph-mon[35962]: pgmap v231: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 685 KiB/s wr, 53 op/s 2024-05-14T17:11:09.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 359941 5.92 MB/sec execute 201 sec latency 2380.787 ms 2024-05-14T17:11:10.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 362119 5.96 MB/sec execute 202 sec latency 2441.941 ms 2024-05-14T17:11:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:10 smithi150 ceph-mon[36204]: pgmap v232: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-14T17:11:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:10 smithi099 ceph-mon[31851]: pgmap v232: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-14T17:11:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:10 smithi193 ceph-mon[35962]: pgmap v232: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-14T17:11:11.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 363483 5.96 MB/sec execute 203 sec latency 118.342 ms 2024-05-14T17:11:12.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 363483 5.93 MB/sec execute 204 sec latency 1118.386 ms 2024-05-14T17:11:13.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:12 smithi150 ceph-mon[36204]: pgmap v233: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-14T17:11:13.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:12 smithi099 ceph-mon[31851]: pgmap v233: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-14T17:11:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:12 smithi193 ceph-mon[35962]: pgmap v233: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-05-14T17:11:13.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 363483 5.90 MB/sec execute 205 sec latency 2118.506 ms 2024-05-14T17:11:14.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 363483 5.87 MB/sec execute 206 sec latency 3118.559 ms 2024-05-14T17:11:15.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:14 smithi150 ceph-mon[36204]: pgmap v234: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-05-14T17:11:15.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:14 smithi150 ceph-mon[36204]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T17:11:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:14 smithi099 ceph-mon[31851]: pgmap v234: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-05-14T17:11:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:14 smithi099 ceph-mon[31851]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T17:11:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:14 smithi193 ceph-mon[35962]: pgmap v234: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-05-14T17:11:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:14 smithi193 ceph-mon[35962]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T17:11:15.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.89 MB/sec execute 207 sec latency 3147.058 ms 2024-05-14T17:11:16.389 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-05-14T17:11:16.389 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 systemd[1]: Stopped Ceph mds.f for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:11:16.389 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service: Consumed 9.101s CPU time. 2024-05-14T17:11:16.390 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 systemd[1]: Starting Ceph mds.f for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:11:16.390 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 podman[72762]: 2024-05-14 17:11:16.172170127 +0000 UTC m=+0.017299432 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:11:16.390 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 podman[72762]: 2024-05-14 17:11:16.283638781 +0000 UTC m=+0.128768086 container create 4114072b90bd0db91612eca1c5c33754a152e0f1d738a05ebfd930bea12f1b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-14T17:11:16.536 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.86 MB/sec execute 208 sec latency 1417.884 ms 2024-05-14T17:11:16.792 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 podman[72762]: 2024-05-14 17:11:16.519516335 +0000 UTC m=+0.364645640 container init 4114072b90bd0db91612eca1c5c33754a152e0f1d738a05ebfd930bea12f1b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:11:16.792 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 podman[72762]: 2024-05-14 17:11:16.522146725 +0000 UTC m=+0.367276031 container start 4114072b90bd0db91612eca1c5c33754a152e0f1d738a05ebfd930bea12f1b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-14T17:11:16.792 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 ceph-mds[72781]: -- 172.21.15.193:0/3023901175 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558382263a40 con 0x558382236c00 2024-05-14T17:11:16.792 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: starting mds.f at 2024-05-14T17:11:16.793 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 bash[72762]: 4114072b90bd0db91612eca1c5c33754a152e0f1d738a05ebfd930bea12f1b00 2024-05-14T17:11:16.793 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:16 smithi193 systemd[1]: Started Ceph mds.f for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:11:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:16 smithi099 ceph-mon[31851]: pgmap v236: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-05-14T17:11:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:16 smithi099 ceph-mon[31851]: daemon mds.f restarted 2024-05-14T17:11:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:16 smithi193 ceph-mon[35962]: pgmap v236: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-05-14T17:11:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:16 smithi193 ceph-mon[35962]: daemon mds.f restarted 2024-05-14T17:11:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:16 smithi150 ceph-mon[36204]: pgmap v236: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-05-14T17:11:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:16 smithi150 ceph-mon[36204]: daemon mds.f restarted 2024-05-14T17:11:17.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.83 MB/sec execute 209 sec latency 2418.012 ms 2024-05-14T17:11:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 -- 172.21.15.99:0/2391498443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088107940 msgr2=0x7f5088107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 --2- 172.21.15.99:0/2391498443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088107940 0x7f5088107d20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5084009cd0 tx=0x7f508402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 -- 172.21.15.99:0/2391498443 shutdown_connections 2024-05-14T17:11:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 --2- 172.21.15.99:0/2391498443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5088102340 0x7f508810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 --2- 172.21.15.99:0/2391498443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088101980 0x7f5088101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 --2- 172.21.15.99:0/2391498443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088107940 0x7f5088107d20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 -- 172.21.15.99:0/2391498443 >> 172.21.15.99:0/2391498443 conn(0x7f5088078ad0 msgr2=0x7f5088078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 -- 172.21.15.99:0/2391498443 shutdown_connections 2024-05-14T17:11:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.679+0000 7f5090104640 1 -- 172.21.15.99:0/2391498443 wait complete. 2024-05-14T17:11:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 Processor -- start 2024-05-14T17:11:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 -- start start 2024-05-14T17:11:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5088101980 0x7f5088199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 0x7f50881996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088107940 0x7f508819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5088111c00 con 0x7f5088102340 2024-05-14T17:11:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5088111a80 con 0x7f5088107940 2024-05-14T17:11:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f5090104640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5088111d80 con 0x7f5088101980 2024-05-14T17:11:17.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f508d69f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 0x7f50881996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f508d69f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 0x7f50881996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43474/0 (socket says 172.21.15.99:43474) 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f508d69f640 1 -- 172.21.15.99:0/515990421 learned_addr learned my addr 172.21.15.99:0/515990421 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f508dea0640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5088101980 0x7f5088199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.680+0000 7f508e6a1640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088107940 0x7f508819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f508d69f640 1 -- 172.21.15.99:0/515990421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5088101980 msgr2=0x7f5088199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f508d69f640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5088101980 0x7f5088199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f508d69f640 1 -- 172.21.15.99:0/515990421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088107940 msgr2=0x7f508819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f508d69f640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088107940 0x7f508819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f508d69f640 1 -- 172.21.15.99:0/515990421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508819e170 con 0x7f5088102340 2024-05-14T17:11:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f508d69f640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 0x7f50881996c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f507800d950 tx=0x7f507800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f507effd640 1 -- 172.21.15.99:0/515990421 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078017030 con 0x7f5088102340 2024-05-14T17:11:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f507effd640 1 -- 172.21.15.99:0/515990421 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5078004020 con 0x7f5088102340 2024-05-14T17:11:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f507effd640 1 -- 172.21.15.99:0/515990421 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50780047a0 con 0x7f5088102340 2024-05-14T17:11:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508819e460 con 0x7f5088102340 2024-05-14T17:11:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.681+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a5cf0 con 0x7f5088102340 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f507effd640 1 -- 172.21.15.99:0/515990421 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5078009a30 con 0x7f5088102340 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 Processor -- start 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 -- start start 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f07bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c034e7ac30 con 0x55c034f077d0 2024-05-14T17:11:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c034e6a7d0 con 0x55c034f16ee0 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f5092e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c034e6e810 con 0x55c034f080f0 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f508e6a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f07bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.684+0000 7f508e6a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f07bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43490/0 (socket says 172.21.15.99:43490) 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 -- 172.21.15.99:0/4228771979 learned_addr learned my addr 172.21.15.99:0/4228771979 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508dea0640 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508d69f640 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f507effd640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f50740b2a10 0x7f50740b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 -- 172.21.15.99:0/4228771979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 msgr2=0x55c034f169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 -- 172.21.15.99:0/4228771979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 msgr2=0x55c034f192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 -- 172.21.15.99:0/4228771979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c034b7f210 con 0x55c034f077d0 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f507effd640 1 -- 172.21.15.99:0/515990421 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8477+0+0 (secure 0 0 0) 0x7f50780e1fc0 con 0x7f5088102340 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f50740b2a10 0x7f50740b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f508e6a1640 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f07bb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5080009c50 tx=0x7f5080011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf33cc134669c974 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.685+0000 7f5073fff640 1 -- 172.21.15.99:0/4228771979 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508001b870 con 0x55c034f077d0 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5073fff640 1 -- 172.21.15.99:0/4228771979 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5080019d80 con 0x55c034f077d0 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5073fff640 1 -- 172.21.15.99:0/4228771979 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508000ecd0 con 0x55c034f077d0 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 -- 172.21.15.99:0/4228771979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 msgr2=0x55c034f07bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f508e6a1640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f50740b2a10 0x7f50740b4ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5080020000 tx=0x7f5080021000 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f07bb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5080009c50 tx=0x7f5080011810 comp rx=0 tx=0).stop 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 -- 172.21.15.99:0/4228771979 shutdown_connections 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 --2- 172.21.15.99:0/4228771979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f07bb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 -- 172.21.15.99:0/4228771979 >> 172.21.15.99:0/4228771979 conn(0x55c034f03e00 msgr2=0x55c034f041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 -- 172.21.15.99:0/4228771979 shutdown_connections 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.686+0000 7f5092e1c740 1 -- 172.21.15.99:0/4228771979 wait complete. 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 -- start start 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 Processor -- start 2024-05-14T17:11:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f16800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f13af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c034b50790 con 0x55c034f16ee0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c034e26890 con 0x55c034f080f0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c034f19bb0 con 0x55c034f077d0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f5092e1c740 10 client.0 _ll_register_callbacks cb 0x55c034e29d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f508e6a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f508e6a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f14030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43500/0 (socket says 172.21.15.99:43500) 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.687+0000 7f508e6a1640 1 -- 172.21.15.99:0/1319326812 learned_addr learned my addr 172.21.15.99:0/1319326812 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508d69f640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f13af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508dea0640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f16800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508e6a1640 1 -- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 msgr2=0x55c034f16800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508e6a1640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f16800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508e6a1640 1 -- 172.21.15.99:0/1319326812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 msgr2=0x55c034f13af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508e6a1640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f13af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508e6a1640 1 -- 172.21.15.99:0/1319326812 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c034c4b620 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508dea0640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f16800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f508e6a1640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f14030 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f508000e950 tx=0x7f508000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ae073bc002d2a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f50717fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f50717fa640 1 -- 172.21.15.99:0/1319326812 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508001b7e0 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f50717fa640 1 -- 172.21.15.99:0/1319326812 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8477+0+0 (secure 0 0 0) 0x7f5080019850 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5080019d80 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f50717fa640 1 -- 172.21.15.99:0/1319326812 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5080019a50 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f5092e1c740 10 client.16212 fetch_fsmap learned FSMap version 31 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c034f25600 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.688+0000 7f50717fa640 1 -- 172.21.15.99:0/1319326812 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5080019d80 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f50717fa640 1 -- 172.21.15.99:0/1319326812 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f5080004710 con 0x55c034f16ee0 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f5092e1c740 10 client.16212 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T17:11:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f5092e1c740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f5092e1c740 20 client.16212 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f5092e1c740 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f26a10 0x55c034f28e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f5092e1c740 4 client.16212 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c034e7cb80 con 0x55c034f26a10 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f508dea0640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f26a10 0x55c034f28e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f50717fa640 10 client.16212 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.689+0000 7f508dea0640 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f26a10 0x55c034f28e00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cebb55149eb379f8 server_cookie=86a5330574de628 in_seq=0 out_seq=0 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f50717fa640 1 -- 172.21.15.99:0/1319326812 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c034e7cb80 con 0x55c034f26a10 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f50717fa640 10 client.16212 handle_command_reply: tid=0 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 client.16212 shutdown 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 msgr2=0x55c034f14030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f14030 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f508000e950 tx=0x7f508000ecc0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 shutdown_connections 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f26a10 0x55c034f28e00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f14030 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f13af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 --2- 172.21.15.99:0/1319326812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f16800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f50717fa640 0 client.16212 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 >> 172.21.15.99:0/1319326812 conn(0x55c034f03e00 msgr2=0x55c034f09c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 shutdown_connections 2024-05-14T17:11:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 1 -- 172.21.15.99:0/1319326812 wait complete. 2024-05-14T17:11:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.690+0000 7f5092e1c740 20 client.16212 trim_cache size 0 max 16384 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.705+0000 7f5092e1c740 1 Processor -- start 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.705+0000 7f5092e1c740 1 -- start start 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f22ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f2e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f5092e1c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50881aed80 con 0x55c034f077d0 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f5092e1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50881ac590 con 0x55c034f16ee0 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f5092e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50881aa3e0 con 0x55c034f080f0 2024-05-14T17:11:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f22ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f22ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43516/0 (socket says 172.21.15.99:43516) 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 -- 172.21.15.99:0/131765586 learned_addr learned my addr 172.21.15.99:0/131765586 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508e6a1640 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 -- 172.21.15.99:0/131765586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 msgr2=0x55c034f0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 -- 172.21.15.99:0/131765586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 msgr2=0x55c034f2e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f2e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 -- 172.21.15.99:0/131765586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c034f03c30 con 0x55c034f077d0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f508dea0640 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f22ac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5084009cd0 tx=0x7f5084009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaf3fa64c3d6a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f50737fe640 1 -- 172.21.15.99:0/131765586 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508404e020 con 0x55c034f077d0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f50737fe640 1 -- 172.21.15.99:0/131765586 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5084036040 con 0x55c034f077d0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.706+0000 7f50737fe640 1 -- 172.21.15.99:0/131765586 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084004040 con 0x55c034f077d0 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 -- 172.21.15.99:0/131765586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 msgr2=0x55c034f22ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f22ac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5084009cd0 tx=0x7f5084009920 comp rx=0 tx=0).stop 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 -- 172.21.15.99:0/131765586 shutdown_connections 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f16ee0 0x55c034f2e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f080f0 0x55c034f0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 --2- 172.21.15.99:0/131765586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f077d0 0x55c034f22ac0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 -- 172.21.15.99:0/131765586 >> 172.21.15.99:0/131765586 conn(0x55c034f03e00 msgr2=0x55c034f0e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 -- 172.21.15.99:0/131765586 shutdown_connections 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 -- 172.21.15.99:0/131765586 wait complete. 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 -- start start 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.707+0000 7f5092e1c740 1 Processor -- start 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f1f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f1acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50881aaa50 con 0x55c034f16ee0 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f50881abec0 con 0x55c034f080f0 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50881ab120 con 0x55c034f077d0 2024-05-14T17:11:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f5092e1c740 10 client.0 _ll_register_callbacks cb 0x7f5080009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f1acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f1acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43522/0 (socket says 172.21.15.99:43522) 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 -- 172.21.15.99:0/613580641 learned_addr learned my addr 172.21.15.99:0/613580641 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508d69f640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508dea0640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f1f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 -- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 msgr2=0x55c034f1f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f1f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 -- 172.21.15.99:0/613580641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 msgr2=0x55c034f1a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f1a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 -- 172.21.15.99:0/613580641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c034f1b500 con 0x55c034f16ee0 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508dea0640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f1f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f508e6a1640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f1acd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5080011850 tx=0x7f508001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d881c142fef661 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f50717fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.708+0000 7f50717fa640 1 -- 172.21.15.99:0/613580641 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080045060 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5084036040 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f50717fa640 1 -- 172.21.15.99:0/613580641 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8477+0+0 (secure 0 0 0) 0x7f5080009780 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f50717fa640 1 -- 172.21.15.99:0/613580641 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5080004680 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 10 client.16218 fetch_fsmap learned FSMap version 31 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c034f91150 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f50717fa640 1 -- 172.21.15.99:0/613580641 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5084036040 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f50717fa640 1 -- 172.21.15.99:0/613580641 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f508003f030 con 0x55c034f16ee0 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 10 client.16218 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 20 client.16218 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f91870 0x55c034f93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 4 client.16218 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c034f143c0 con 0x55c034f91870 2024-05-14T17:11:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.709+0000 7f508dea0640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f91870 0x55c034f93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.710+0000 7f50717fa640 10 client.16218 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.710+0000 7f508dea0640 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f91870 0x55c034f93c90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44a154c6653c845 server_cookie=400c6bdedecb521c in_seq=0 out_seq=0 2024-05-14T17:11:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.740+0000 7f50717fa640 1 -- 172.21.15.99:0/613580641 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c034f143c0 con 0x55c034f91870 2024-05-14T17:11:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f50717fa640 10 client.16218 handle_command_reply: tid=0 2024-05-14T17:11:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 client.16218 shutdown 2024-05-14T17:11:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 msgr2=0x55c034f1acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f1acd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5080011850 tx=0x7f508001ba10 comp rx=0 tx=0).stop 2024-05-14T17:11:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 shutdown_connections 2024-05-14T17:11:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c034f91870 0x55c034f93c90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c034f16ee0 0x55c034f1acd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c034f080f0 0x55c034f1a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 --2- 172.21.15.99:0/613580641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c034f077d0 0x55c034f1f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f50717fa640 0 client.16218 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 >> 172.21.15.99:0/613580641 conn(0x55c034f03e00 msgr2=0x55c034f0f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 shutdown_connections 2024-05-14T17:11:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 1 -- 172.21.15.99:0/613580641 wait complete. 2024-05-14T17:11:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.741+0000 7f5092e1c740 20 client.16218 trim_cache size 0 max 16384 2024-05-14T17:11:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:11:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:11:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:11:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f50740b2a10 msgr2=0x7f50740b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f50740b2a10 0x7f50740b4ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5080020000 tx=0x7f5080021000 comp rx=0 tx=0).stop 2024-05-14T17:11:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 msgr2=0x7f50881996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 0x7f50881996c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f507800d950 tx=0x7f507800de10 comp rx=0 tx=0).stop 2024-05-14T17:11:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 shutdown_connections 2024-05-14T17:11:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f50740b2a10 0x7f50740b4ed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088107940 0x7f508819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5088102340 0x7f50881996c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 --2- 172.21.15.99:0/515990421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5088101980 0x7f5088199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 >> 172.21.15.99:0/515990421 conn(0x7f5088078ad0 msgr2=0x7f508806a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 shutdown_connections 2024-05-14T17:11:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:17.742+0000 7f5090104640 1 -- 172.21.15.99:0/515990421 wait complete. 2024-05-14T17:11:18.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:17 smithi099 ceph-mon[31851]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T17:11:18.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:17 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6834/3434670895,v1:172.21.15.193:6835/3434670895] up:boot 2024-05-14T17:11:18.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:17 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:11:18.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:17 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:11:18.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:17 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:18.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:17 smithi193 ceph-mon[35962]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T17:11:18.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:17 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6834/3434670895,v1:172.21.15.193:6835/3434670895] up:boot 2024-05-14T17:11:18.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:17 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:11:18.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:17 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:11:18.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:17 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:17 smithi150 ceph-mon[36204]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T17:11:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:17 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6834/3434670895,v1:172.21.15.193:6835/3434670895] up:boot 2024-05-14T17:11:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:17 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:11:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:17 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:11:18.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:17 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:18.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.80 MB/sec execute 210 sec latency 3418.105 ms 2024-05-14T17:11:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:18 smithi099 ceph-mon[31851]: pgmap v238: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 742 KiB/s wr, 65 op/s 2024-05-14T17:11:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:18 smithi193 ceph-mon[35962]: pgmap v238: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 742 KiB/s wr, 65 op/s 2024-05-14T17:11:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:18 smithi150 ceph-mon[36204]: pgmap v238: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 742 KiB/s wr, 65 op/s 2024-05-14T17:11:19.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.78 MB/sec execute 211 sec latency 4418.146 ms 2024-05-14T17:11:20.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.75 MB/sec execute 212 sec latency 5418.198 ms 2024-05-14T17:11:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:20 smithi099 ceph-mon[31851]: pgmap v239: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-05-14T17:11:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:20 smithi193 ceph-mon[35962]: pgmap v239: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-05-14T17:11:21.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:20 smithi150 ceph-mon[36204]: pgmap v239: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-05-14T17:11:21.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.72 MB/sec execute 213 sec latency 6418.267 ms 2024-05-14T17:11:22.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.69 MB/sec execute 214 sec latency 7418.327 ms 2024-05-14T17:11:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:22 smithi099 ceph-mon[31851]: pgmap v240: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 323 KiB/s wr, 53 op/s 2024-05-14T17:11:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:22 smithi193 ceph-mon[35962]: pgmap v240: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 323 KiB/s wr, 53 op/s 2024-05-14T17:11:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:22 smithi150 ceph-mon[36204]: pgmap v240: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 323 KiB/s wr, 53 op/s 2024-05-14T17:11:23.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.67 MB/sec execute 215 sec latency 8418.389 ms 2024-05-14T17:11:24.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 365064 5.64 MB/sec execute 216 sec latency 9418.482 ms 2024-05-14T17:11:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:24 smithi099 ceph-mon[31851]: pgmap v241: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 720 KiB/s wr, 72 op/s 2024-05-14T17:11:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:24 smithi193 ceph-mon[35962]: pgmap v241: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 720 KiB/s wr, 72 op/s 2024-05-14T17:11:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:24 smithi150 ceph-mon[36204]: pgmap v241: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 720 KiB/s wr, 72 op/s 2024-05-14T17:11:25.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 366793 5.66 MB/sec execute 217 sec latency 9476.159 ms 2024-05-14T17:11:26.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 367032 5.64 MB/sec execute 218 sec latency 856.121 ms 2024-05-14T17:11:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:26 smithi099 ceph-mon[31851]: pgmap v242: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 636 KiB/s wr, 63 op/s 2024-05-14T17:11:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:26 smithi193 ceph-mon[35962]: pgmap v242: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 636 KiB/s wr, 63 op/s 2024-05-14T17:11:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:26 smithi150 ceph-mon[36204]: pgmap v242: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 636 KiB/s wr, 63 op/s 2024-05-14T17:11:27.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 367032 5.61 MB/sec execute 219 sec latency 1856.213 ms 2024-05-14T17:11:28.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 367032 5.58 MB/sec execute 220 sec latency 2856.322 ms 2024-05-14T17:11:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:28 smithi099 ceph-mon[31851]: pgmap v243: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 592 KiB/s wr, 59 op/s 2024-05-14T17:11:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:28 smithi193 ceph-mon[35962]: pgmap v243: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 592 KiB/s wr, 59 op/s 2024-05-14T17:11:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:28 smithi150 ceph-mon[36204]: pgmap v243: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 592 KiB/s wr, 59 op/s 2024-05-14T17:11:29.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 367032 5.56 MB/sec execute 221 sec latency 3856.429 ms 2024-05-14T17:11:30.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 369321 5.60 MB/sec execute 222 sec latency 3916.061 ms 2024-05-14T17:11:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 -- 172.21.15.99:0/3552146012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7570103d00 msgr2=0x7f75701040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 --2- 172.21.15.99:0/3552146012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7570103d00 0x7f75701040e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7560009900 tx=0x7f756002a480 comp rx=0 tx=0).stop 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 -- 172.21.15.99:0/3552146012 shutdown_connections 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 --2- 172.21.15.99:0/3552146012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757010d0d0 0x7f757010f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 --2- 172.21.15.99:0/3552146012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570104620 0x7f757010cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 --2- 172.21.15.99:0/3552146012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7570103d00 0x7f75701040e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 -- 172.21.15.99:0/3552146012 >> 172.21.15.99:0/3552146012 conn(0x7f7570077cd0 msgr2=0x7f75700780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 -- 172.21.15.99:0/3552146012 shutdown_connections 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 -- 172.21.15.99:0/3552146012 wait complete. 2024-05-14T17:11:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.834+0000 7f7575114640 1 Processor -- start 2024-05-14T17:11:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 -- start start 2024-05-14T17:11:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 0x7f7570101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7570104620 0x7f7570101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 0x7f75701023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112d10 con 0x7f757010d0d0 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112b90 con 0x7f7570104620 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f7575114640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112e90 con 0x7f7570103d00 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 0x7f7570101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 0x7f75701023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 0x7f7570101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:55316/0 (socket says 172.21.15.99:55316) 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 -- 172.21.15.99:0/3943251978 learned_addr learned my addr 172.21.15.99:0/3943251978 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 0x7f75701023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36242/0 (socket says 172.21.15.99:36242) 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756e59c640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7570104620 0x7f7570101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 -- 172.21.15.99:0/3943251978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7570104620 msgr2=0x7f7570101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7570104620 0x7f7570101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 -- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 msgr2=0x7f75701023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 0x7f75701023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756ed9d640 1 -- 172.21.15.99:0/3943251978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701a76f0 con 0x7f7570103d00 2024-05-14T17:11:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.835+0000 7f756f59e640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 0x7f75701023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.836+0000 7f756ed9d640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 0x7f7570101950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7560005940 tx=0x7f75600040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.836+0000 7f7557fff640 1 -- 172.21.15.99:0/3943251978 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756003f030 con 0x7f7570103d00 2024-05-14T17:11:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.836+0000 7f7557fff640 1 -- 172.21.15.99:0/3943251978 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75600044c0 con 0x7f7570103d00 2024-05-14T17:11:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.836+0000 7f7557fff640 1 -- 172.21.15.99:0/3943251978 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560035070 con 0x7f7570103d00 2024-05-14T17:11:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.836+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701a7980 con 0x7f7570103d00 2024-05-14T17:11:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.836+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a7e70 con 0x7f7570103d00 2024-05-14T17:11:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.838+0000 7f7577e2f740 1 Processor -- start 2024-05-14T17:11:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.838+0000 7f7577e2f740 1 -- start start 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 0x56039e7b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7bdad0 0x56039e7bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7577e2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56039e725c30 con 0x56039e7b3210 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7577e2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56039e7157d0 con 0x56039e7bdad0 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7577e2f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56039e719810 con 0x56039e7b28f0 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 0x56039e7b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 0x56039e7b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:55328/0 (socket says 172.21.15.99:55328) 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 -- 172.21.15.99:0/4176648402 learned_addr learned my addr 172.21.15.99:0/4176648402 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756e59c640 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7bdad0 0x56039e7bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 -- 172.21.15.99:0/4176648402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7bdad0 msgr2=0x56039e7bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7bdad0 0x56039e7bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 -- 172.21.15.99:0/4176648402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 msgr2=0x56039e7bd590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7bd590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 -- 172.21.15.99:0/4176648402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039e42a210 con 0x56039e7b28f0 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756e59c640 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7bdad0 0x56039e7bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f7557fff640 1 -- 172.21.15.99:0/3943251978 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7560042030 con 0x7f7570103d00 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.839+0000 7f756f59e640 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 0x56039e7b2cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7568011760 tx=0x7f7568011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ba5f246892a71b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f75557fa640 1 -- 172.21.15.99:0/4176648402 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568004d20 con 0x56039e7b28f0 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f75557fa640 1 -- 172.21.15.99:0/4176648402 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f756800f9a0 con 0x56039e7b28f0 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f75557fa640 1 -- 172.21.15.99:0/4176648402 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568004340 con 0x56039e7b28f0 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f7557fff640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f75580b2a10 0x7f75580b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f756ed9d640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f75580b2a10 0x7f75580b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f7557fff640 1 -- 172.21.15.99:0/3943251978 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8477+0+0 (secure 0 0 0) 0x7f7560102960 con 0x7f7570103d00 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f7577e2f740 1 -- 172.21.15.99:0/4176648402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 msgr2=0x56039e7b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.840+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 0x56039e7b2cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7568011760 tx=0x7f7568011c20 comp rx=0 tx=0).stop 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- 172.21.15.99:0/4176648402 shutdown_connections 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7bdad0 0x56039e7bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4176648402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7b28f0 0x56039e7b2cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- 172.21.15.99:0/4176648402 >> 172.21.15.99:0/4176648402 conn(0x56039e7aee00 msgr2=0x56039e7af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- 172.21.15.99:0/4176648402 shutdown_connections 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- 172.21.15.99:0/4176648402 wait complete. 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f756ed9d640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f75580b2a10 0x7f75580b4ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56039e7b3e90 tx=0x7f7560049dd0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- start start 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 Processor -- start 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7ba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7badf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56039e3fb790 con 0x56039e7b3210 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56039e6d1890 con 0x56039e7b28f0 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56039e7c07a0 con 0x56039e7bdad0 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.841+0000 7f7577e2f740 10 client.0 _ll_register_callbacks cb 0x56039e6d4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7ba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7bd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:35808/0 (socket says 172.21.15.99:35808) 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 -- 172.21.15.99:0/952920290 learned_addr learned my addr 172.21.15.99:0/952920290 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7badf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 -- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 msgr2=0x56039e7badf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7badf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 -- 172.21.15.99:0/952920290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 msgr2=0x56039e7ba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7ba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 -- 172.21.15.99:0/952920290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039e4f6620 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756e59c640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f756f59e640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7bd4f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f756800f790 tx=0x7f7568004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda7ffce2a7a2175 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7552ff5640 1 -- 172.21.15.99:0/952920290 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756801e040 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756800f9a0 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7552ff5640 1 -- 172.21.15.99:0/952920290 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8477+0+0 (secure 0 0 0) 0x7f7568011220 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7552ff5640 1 -- 172.21.15.99:0/952920290 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f75680042d0 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7577e2f740 10 client.25544 fetch_fsmap learned FSMap version 31 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.842+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56039e7cc280 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7552ff5640 1 -- 172.21.15.99:0/952920290 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f756800f9a0 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7552ff5640 1 -- 172.21.15.99:0/952920290 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7568024070 con 0x56039e7b28f0 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7577e2f740 10 client.25544 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T17:11:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7577e2f740 10 client.25544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7577e2f740 20 client.25544 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7577e2f740 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e7cd6f0 0x56039e7cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7577e2f740 4 client.25544 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56039e727b80 con 0x56039e7cd6f0 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f756e59c640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e7cd6f0 0x56039e7cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f7552ff5640 10 client.25544 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.843+0000 7f756e59c640 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e7cd6f0 0x56039e7cfae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11173967d341f69 server_cookie=a0e3006faad9e0f3 in_seq=0 out_seq=0 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7552ff5640 1 -- 172.21.15.99:0/952920290 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56039e727b80 con 0x56039e7cd6f0 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7552ff5640 10 client.25544 handle_command_reply: tid=0 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 client.25544 shutdown 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 msgr2=0x56039e7bd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7bd4f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f756800f790 tx=0x7f7568004f60 comp rx=0 tx=0).stop 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 shutdown_connections 2024-05-14T17:11:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e7cd6f0 0x56039e7cfae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7badf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7ba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7552ff5640 0 client.25544 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 --2- 172.21.15.99:0/952920290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7bd4f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 >> 172.21.15.99:0/952920290 conn(0x56039e7aee00 msgr2=0x56039e7ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 shutdown_connections 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 1 -- 172.21.15.99:0/952920290 wait complete. 2024-05-14T17:11:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.844+0000 7f7577e2f740 20 client.25544 trim_cache size 0 max 16384 2024-05-14T17:11:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 Processor -- start 2024-05-14T17:11:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 -- start start 2024-05-14T17:11:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75701b3130 con 0x56039e7b3210 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75701b0940 con 0x56039e7b28f0 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f7577e2f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75701ae790 con 0x56039e7bdad0 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7d5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36254/0 (socket says 172.21.15.99:36254) 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 -- 172.21.15.99:0/4141991971 learned_addr learned my addr 172.21.15.99:0/4141991971 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756f59e640 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756f59e640 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 -- 172.21.15.99:0/4141991971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 msgr2=0x56039e7d57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7d57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 -- 172.21.15.99:0/4141991971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 msgr2=0x56039e7c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 -- 172.21.15.99:0/4141991971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039e7d0040 con 0x56039e7b3210 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756f59e640 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.860+0000 7f756e59c640 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7d5290 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x56039e7aff90 tx=0x7f7564005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef14ccb1c0c5c72e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7554ff9640 1 -- 172.21.15.99:0/4141991971 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564004030 con 0x56039e7b3210 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7554ff9640 1 -- 172.21.15.99:0/4141991971 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7564005020 con 0x56039e7b3210 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7554ff9640 1 -- 172.21.15.99:0/4141991971 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400fa70 con 0x56039e7b3210 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 -- 172.21.15.99:0/4141991971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 msgr2=0x56039e7d5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7d5290 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x56039e7aff90 tx=0x7f7564005f90 comp rx=0 tx=0).stop 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 -- 172.21.15.99:0/4141991971 shutdown_connections 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7d57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b3210 0x56039e7d5290 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 --2- 172.21.15.99:0/4141991971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b28f0 0x56039e7c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 -- 172.21.15.99:0/4141991971 >> 172.21.15.99:0/4141991971 conn(0x56039e7aee00 msgr2=0x56039e7ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.861+0000 7f7577e2f740 1 -- 172.21.15.99:0/4141991971 shutdown_connections 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 -- 172.21.15.99:0/4141991971 wait complete. 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 -- start start 2024-05-14T17:11:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 Processor -- start 2024-05-14T17:11:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 0x56039e7c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b3210 0x56039e7c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75701aee00 con 0x56039e7b28f0 2024-05-14T17:11:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75701b0270 con 0x56039e7b3210 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75701af4d0 con 0x56039e7bdad0 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f7577e2f740 10 client.0 _ll_register_callbacks cb 0x7f7568003a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f756f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 0x56039e7c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f756f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 0x56039e7c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36266/0 (socket says 172.21.15.99:36266) 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f756f59e640 1 -- 172.21.15.99:0/1046809095 learned_addr learned my addr 172.21.15.99:0/1046809095 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f756e59c640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b3210 0x56039e7c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.862+0000 7f756f59e640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f756f59e640 1 -- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 msgr2=0x56039e7c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f756f59e640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f756f59e640 1 -- 172.21.15.99:0/1046809095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b3210 msgr2=0x56039e7c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f756f59e640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b3210 0x56039e7c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f756f59e640 1 -- 172.21.15.99:0/1046809095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039e7c3f80 con 0x56039e7b28f0 2024-05-14T17:11:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f756f59e640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 0x56039e7c5ee0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7568005600 tx=0x7f7568005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6230c13b552d3dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7552ff5640 1 -- 172.21.15.99:0/1046809095 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568004fd0 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7552ff5640 1 -- 172.21.15.99:0/1046809095 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8477+0+0 (secure 0 0 0) 0x7f7568005170 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7564005020 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7552ff5640 1 -- 172.21.15.99:0/1046809095 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7568004470 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 10 client.16233 fetch_fsmap learned FSMap version 31 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56039e837d70 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7552ff5640 1 -- 172.21.15.99:0/1046809095 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7564005020 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7552ff5640 1 -- 172.21.15.99:0/1046809095 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f7568024040 con 0x56039e7b28f0 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 10 client.16233 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 20 client.16233 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e838400 0x56039e83a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 4 client.16233 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.863+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56039e721360 con 0x56039e838400 2024-05-14T17:11:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.864+0000 7f756e59c640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e838400 0x56039e83a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.864+0000 7f756e59c640 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e838400 0x56039e83a820 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd653faecef7077 server_cookie=40da9a091c046ab9 in_seq=0 out_seq=0 2024-05-14T17:11:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.864+0000 7f7552ff5640 10 client.16233 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7552ff5640 1 -- 172.21.15.99:0/1046809095 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56039e721360 con 0x56039e838400 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7552ff5640 10 client.16233 handle_command_reply: tid=0 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 client.16233 shutdown 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 msgr2=0x56039e7c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 0x56039e7c5ee0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7568005600 tx=0x7f7568005bd0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 shutdown_connections 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56039e838400 0x56039e83a820 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56039e7bdad0 0x56039e7c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56039e7b3210 0x56039e7c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 --2- 172.21.15.99:0/1046809095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56039e7b28f0 0x56039e7c5ee0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 >> 172.21.15.99:0/1046809095 conn(0x56039e7aee00 msgr2=0x56039e7c8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7552ff5640 0 client.16233 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 shutdown_connections 2024-05-14T17:11:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 1 -- 172.21.15.99:0/1046809095 wait complete. 2024-05-14T17:11:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.899+0000 7f7577e2f740 20 client.16233 trim_cache size 0 max 16384 2024-05-14T17:11:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:11:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:11:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f75580b2a10 msgr2=0x7f75580b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f75580b2a10 0x7f75580b4ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56039e7b3e90 tx=0x7f7560049dd0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 msgr2=0x7f7570101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 0x7f7570101950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7560005940 tx=0x7f75600040c0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 shutdown_connections 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f75580b2a10 0x7f75580b4ed0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f757010d0d0 0x7f75701023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7570104620 0x7f7570101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 --2- 172.21.15.99:0/3943251978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570103d00 0x7f7570101950 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 >> 172.21.15.99:0/3943251978 conn(0x7f7570077cd0 msgr2=0x7f75700ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 shutdown_connections 2024-05-14T17:11:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:30.900+0000 7f7575114640 1 -- 172.21.15.99:0/3943251978 wait complete. 2024-05-14T17:11:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:30 smithi099 ceph-mon[31851]: pgmap v244: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 742 KiB/s wr, 81 op/s 2024-05-14T17:11:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:30 smithi193 ceph-mon[35962]: pgmap v244: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 742 KiB/s wr, 81 op/s 2024-05-14T17:11:31.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: ignoring --setuser ceph since I am not root 2024-05-14T17:11:31.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: ignoring --setgroup ceph since I am not root 2024-05-14T17:11:31.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:31 smithi193 ceph-mds[72781]: -- 172.21.15.193:0/3711928006 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3ab839a40 con 0x55b3ab80d000 2024-05-14T17:11:31.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:11:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: starting mds.f at 2024-05-14T17:11:31.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:30 smithi150 ceph-mon[36204]: pgmap v244: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 742 KiB/s wr, 81 op/s 2024-05-14T17:11:31.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 370575 5.60 MB/sec execute 223 sec latency 239.508 ms 2024-05-14T17:11:32.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:32 smithi099 ceph-mon[31851]: daemon mds.f restarted 2024-05-14T17:11:32.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:32 smithi193 ceph-mon[35962]: daemon mds.f restarted 2024-05-14T17:11:32.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:32 smithi150 ceph-mon[36204]: daemon mds.f restarted 2024-05-14T17:11:32.537 INFO:tasks.workunit.client.0.smithi099.stdout: 1 370575 5.57 MB/sec execute 224 sec latency 1239.546 ms 2024-05-14T17:11:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:33 smithi099 ceph-mon[31851]: pgmap v245: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 526 KiB/s wr, 46 op/s 2024-05-14T17:11:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:33 smithi099 ceph-mon[31851]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T17:11:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:33 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6834/4018228315,v1:172.21.15.193:6835/4018228315] up:boot 2024-05-14T17:11:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:33 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:11:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:33 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:11:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:33 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:33 smithi193 ceph-mon[35962]: pgmap v245: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 526 KiB/s wr, 46 op/s 2024-05-14T17:11:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:33 smithi193 ceph-mon[35962]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T17:11:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:33 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6834/4018228315,v1:172.21.15.193:6835/4018228315] up:boot 2024-05-14T17:11:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:33 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:11:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:33 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:11:33.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:33 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:33 smithi150 ceph-mon[36204]: pgmap v245: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 526 KiB/s wr, 46 op/s 2024-05-14T17:11:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:33 smithi150 ceph-mon[36204]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T17:11:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:33 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6834/4018228315,v1:172.21.15.193:6835/4018228315] up:boot 2024-05-14T17:11:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:33 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:11:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:33 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:11:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:33 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:33.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 370575 5.55 MB/sec execute 225 sec latency 2239.637 ms 2024-05-14T17:11:34.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 370575 5.52 MB/sec execute 226 sec latency 3239.734 ms 2024-05-14T17:11:35.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:35 smithi099 ceph-mon[31851]: pgmap v247: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:35 smithi193 ceph-mon[35962]: pgmap v247: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:35.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:35 smithi150 ceph-mon[36204]: pgmap v247: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:35.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 372823 5.56 MB/sec execute 227 sec latency 3297.738 ms 2024-05-14T17:11:36.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:36 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:36.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:36 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:36.342 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.3 2024-05-14T17:11:36.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:11:36.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:36 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:36.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 374123 5.56 MB/sec execute 228 sec latency 155.034 ms 2024-05-14T17:11:36.588 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:37.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:37 smithi099 ceph-mon[31851]: pgmap v248: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:37 smithi193 ceph-mon[35962]: pgmap v248: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:37 smithi150 ceph-mon[36204]: pgmap v248: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:37.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 374123 5.53 MB/sec execute 229 sec latency 1155.103 ms 2024-05-14T17:11:38.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 -- 172.21.15.99:0/4170277733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 msgr2=0x7f79e8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 --2- 172.21.15.99:0/4170277733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8109990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009990 tx=0x7f79dc009ca0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 -- 172.21.15.99:0/4170277733 shutdown_connections 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 --2- 172.21.15.99:0/4170277733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79e806bbe0 0x7f79e81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 --2- 172.21.15.99:0/4170277733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8109990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 --2- 172.21.15.99:0/4170277733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 0x7f79e806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 -- 172.21.15.99:0/4170277733 >> 172.21.15.99:0/4170277733 conn(0x7f79e80793f0 msgr2=0x7f79e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.294+0000 7f79ef86e700 1 -- 172.21.15.99:0/4170277733 shutdown_connections 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 -- 172.21.15.99:0/4170277733 wait complete. 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 Processor -- start 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 -- start start 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 0x7f79e819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79e806bbe0 0x7f79e819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:38.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81136e0 con 0x7f79e81095b0 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8113560 con 0x7f79e806b220 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.295+0000 7f79ef86e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8113860 con 0x7f79e806bbe0 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53932/0 (socket says 172.21.15.99:53932) 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 -- 172.21.15.99:0/88895458 learned_addr learned my addr 172.21.15.99:0/88895458 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ece09700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79e806bbe0 0x7f79e819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ed60a700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 0x7f79e819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 -- 172.21.15.99:0/88895458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79e806bbe0 msgr2=0x7f79e819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79e806bbe0 0x7f79e819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 -- 172.21.15.99:0/88895458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 msgr2=0x7f79e819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 0x7f79e819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 -- 172.21.15.99:0/88895458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e819a0e0 con 0x7f79e81095b0 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ed60a700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 0x7f79e819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:38.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ede0b700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8199980 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f79e0007ee0 tx=0x7f79e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e000dbd0 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79e000dd70 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0005990 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e819a3d0 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.296+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e81a6690 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.298+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e810ba90 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.299+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f79e001a020 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.300+0000 7f79da7fc700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f79d40b2a70 0x7f79d40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.300+0000 7f79ed60a700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f79d40b2a70 0x7f79d40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.300+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f79e00e1c80 con 0x7f79e81095b0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.300+0000 7f79ed60a700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f79d40b2a70 0x7f79d40b4f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006010 tx=0x7f79dc00b540 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:38.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.304+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79e0010070 con 0x7f79e81095b0 2024-05-14T17:11:38.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 374123 5.51 MB/sec execute 230 sec latency 2155.147 ms 2024-05-14T17:11:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.561+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e806a230 con 0x7f79e81095b0 2024-05-14T17:11:38.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.562+0000 7f79da7fc700 1 -- 172.21.15.99:0/88895458 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11876 (secure 0 0 0) 0x7f79e00a9690 con 0x7f79e81095b0 2024-05-14T17:11:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:11:38.562 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:11:35.000334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f79d40b2a70 msgr2=0x7f79d40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f79d40b2a70 0x7f79d40b4f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006010 tx=0x7f79dc00b540 comp rx=0 tx=0).stop 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 msgr2=0x7f79e8199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8199980 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f79e0007ee0 tx=0x7f79e000d3b0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 shutdown_connections 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f79d40b2a70 0x7f79d40b4f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79e81095b0 0x7f79e8199980 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79e806b220 0x7f79e819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 --2- 172.21.15.99:0/88895458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79e806bbe0 0x7f79e819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 >> 172.21.15.99:0/88895458 conn(0x7f79e80793f0 msgr2=0x7f79e8101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 shutdown_connections 2024-05-14T17:11:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:38.564+0000 7f79ef86e700 1 -- 172.21.15.99:0/88895458 wait complete. 2024-05-14T17:11:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 34 2024-05-14T17:11:39.129 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:39 smithi099 ceph-mon[31851]: pgmap v249: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:39.129 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:39 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/88895458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:39.156 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 151.3 --await-for 45.0' 2024-05-14T17:11:39.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 151.3 --await-for 45.0 2024-05-14T17:11:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:39 smithi193 ceph-mon[35962]: pgmap v249: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:39 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/88895458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:39.415 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:11:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:39 smithi150 ceph-mon[36204]: pgmap v249: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 645 KiB/s wr, 105 op/s 2024-05-14T17:11:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:39 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/88895458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:11:39.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 374123 5.48 MB/sec execute 231 sec latency 3155.173 ms 2024-05-14T17:11:40.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 376409 5.52 MB/sec execute 232 sec latency 3182.534 ms 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 -- 172.21.15.99:0/2091493997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 msgr2=0x7f9da41088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 --2- 172.21.15.99:0/2091493997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41088a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009990 tx=0x7f9d94009ca0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 -- 172.21.15.99:0/2091493997 shutdown_connections 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 --2- 172.21.15.99:0/2091493997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da4102e00 0x7f9da410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 --2- 172.21.15.99:0/2091493997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41088a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 --2- 172.21.15.99:0/2091493997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9da4102440 0x7f9da41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 -- 172.21.15.99:0/2091493997 >> 172.21.15.99:0/2091493997 conn(0x7f9da40793e0 msgr2=0x7f9da40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 -- 172.21.15.99:0/2091493997 shutdown_connections 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.063+0000 7f9daa986700 1 -- 172.21.15.99:0/2091493997 wait complete. 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 Processor -- start 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 -- start start 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da4102440 0x7f9da419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9da4102e00 0x7f9da419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4112640 con 0x7f9da41084c0 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41124c0 con 0x7f9da4102e00 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9daa986700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41127c0 con 0x7f9da4102440 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53950/0 (socket says 172.21.15.99:53950) 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da4102440 0x7f9da419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 -- 172.21.15.99:0/344680394 learned_addr learned my addr 172.21.15.99:0/344680394 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9da4102e00 0x7f9da419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 -- 172.21.15.99:0/344680394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da4102440 msgr2=0x7f9da419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da4102440 0x7f9da419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 -- 172.21.15.99:0/344680394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9da4102e00 msgr2=0x7f9da419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9da4102e00 0x7f9da419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 -- 172.21.15.99:0/344680394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da41a5fa0 con 0x7f9da41084c0 2024-05-14T17:11:41.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.064+0000 7f9da8f23700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41a5a60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ef30 tx=0x7f9d9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.065+0000 7f9da1ffb700 1 -- 172.21.15.99:0/344680394 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0092e0 con 0x7f9da41084c0 2024-05-14T17:11:41.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.065+0000 7f9da1ffb700 1 -- 172.21.15.99:0/344680394 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9c004bb0 con 0x7f9da41084c0 2024-05-14T17:11:41.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.065+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da41a6170 con 0x7f9da41084c0 2024-05-14T17:11:41.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.065+0000 7f9da1ffb700 1 -- 172.21.15.99:0/344680394 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c004020 con 0x7f9da41084c0 2024-05-14T17:11:41.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.065+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a66b0 con 0x7f9da41084c0 2024-05-14T17:11:41.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.067+0000 7f9da1ffb700 1 -- 172.21.15.99:0/344680394 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9d9c01b030 con 0x7f9da41084c0 2024-05-14T17:11:41.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.068+0000 7f9da1ffb700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9d900b2aa0 0x7f9d900b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.068+0000 7f9da3fff700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9d900b2aa0 0x7f9d900b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.068+0000 7f9da1ffb700 1 -- 172.21.15.99:0/344680394 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f9d9c0e2000 con 0x7f9da41084c0 2024-05-14T17:11:41.068 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:41 smithi099 ceph-mon[31851]: pgmap v250: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 754 KiB/s wr, 114 op/s 2024-05-14T17:11:41.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.068+0000 7f9db521cb80 1 Processor -- start 2024-05-14T17:11:41.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 -- start start 2024-05-14T17:11:41.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da3fff700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9d900b2aa0 0x7f9d900b4f60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b2e0 tx=0x7f9d94005fd0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641db4b0 0x5580641db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 0x5580641d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558064134f00 con 0x5580641d7350 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558064124aa0 con 0x5580641db4b0 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9db521cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558064128ae0 con 0x5580641dbdd0 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641db4b0 0x5580641db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 0x5580641d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641db4b0 0x5580641db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:51458/0 (socket says 172.21.15.99:51458) 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da3fff700 1 -- 172.21.15.99:0/1201879584 learned_addr learned my addr 172.21.15.99:0/1201879584 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 0x5580641d77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53958/0 (socket says 172.21.15.99:53958) 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9d9bfff700 1 -- 172.21.15.99:0/1201879584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641db4b0 msgr2=0x5580641db890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641db4b0 0x5580641db890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9d9bfff700 1 -- 172.21.15.99:0/1201879584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 msgr2=0x5580641d77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 0x5580641d77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9d9bfff700 1 -- 172.21.15.99:0/1201879584 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580640ce140 con 0x5580641dbdd0 2024-05-14T17:11:41.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.069+0000 7f9da8f23700 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 0x5580641d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:41.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.070+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641d6d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9da406bd90 tx=0x7f9d8c007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba7934b1ed24f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.070+0000 7f9d9a7fc700 1 -- 172.21.15.99:0/1201879584 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c004310 con 0x5580641dbdd0 2024-05-14T17:11:41.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.070+0000 7f9d9a7fc700 1 -- 172.21.15.99:0/1201879584 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c015070 con 0x5580641dbdd0 2024-05-14T17:11:41.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.070+0000 7f9d9a7fc700 1 -- 172.21.15.99:0/1201879584 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c004d00 con 0x5580641dbdd0 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 -- 172.21.15.99:0/1201879584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 msgr2=0x5580641d6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641d6d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9da406bd90 tx=0x7f9d8c007ea0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 -- 172.21.15.99:0/1201879584 shutdown_connections 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641d7350 0x5580641d77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641db4b0 0x5580641db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 --2- 172.21.15.99:0/1201879584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641d6d30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 -- 172.21.15.99:0/1201879584 >> 172.21.15.99:0/1201879584 conn(0x558064103f40 msgr2=0x5580641d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 -- 172.21.15.99:0/1201879584 shutdown_connections 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 -- 172.21.15.99:0/1201879584 wait complete. 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.071+0000 7f9db521cb80 1 -- start start 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 Processor -- start 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641db4b0 0x5580641dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55806402b130 con 0x5580641db4b0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55806408cf30 con 0x5580641d7350 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5580641e3910 con 0x5580641dbdd0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9db521cb80 10 client.0 _ll_register_callbacks cb 0x5580640bfae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9d9bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641db4b0 0x5580641dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9d9bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641db4b0 0x5580641dfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53972/0 (socket says 172.21.15.99:53972) 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9d9bfff700 1 -- 172.21.15.99:0/3487883657 learned_addr learned my addr 172.21.15.99:0/3487883657 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da3fff700 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da8f23700 1 -- 172.21.15.99:0/3487883657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 msgr2=0x5580641e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da8f23700 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da8f23700 1 -- 172.21.15.99:0/3487883657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641db4b0 msgr2=0x5580641dfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da8f23700 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641db4b0 0x5580641dfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.072+0000 7f9da8f23700 1 -- 172.21.15.99:0/3487883657 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558064074a60 con 0x5580641dbdd0 2024-05-14T17:11:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9da8f23700 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641e0530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0df780 tx=0x7f9d9c0a98e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e45b1796a019f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9d837fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9d837fe700 1 -- 172.21.15.99:0/3487883657 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0c3060 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c015070 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9d837fe700 1 -- 172.21.15.99:0/3487883657 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f9d9c0a8810 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9d837fe700 1 -- 172.21.15.99:0/3487883657 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c0a93a0 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9db521cb80 10 client.25705 fetch_fsmap learned FSMap version 34 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5580640de8f0 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9d837fe700 1 -- 172.21.15.99:0/3487883657 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c015070 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9d837fe700 1 -- 172.21.15.99:0/3487883657 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9d9c0a8070 con 0x5580641dbdd0 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9db521cb80 10 client.25705 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9db521cb80 10 client.25705 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.073+0000 7f9db521cb80 20 client.25705 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.074+0000 7f9db521cb80 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5580641f04e0 0x5580641f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.074+0000 7f9db521cb80 4 client.25705 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:41.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.074+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558064136e50 con 0x5580641f04e0 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.074+0000 7f9da3fff700 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5580641f04e0 0x5580641f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.074+0000 7f9d837fe700 10 client.25705 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.074+0000 7f9da3fff700 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5580641f04e0 0x5580641f28d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752d0427e8256c00 server_cookie=47a9faf5328f39ed in_seq=0 out_seq=0 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9d837fe700 1 -- 172.21.15.99:0/3487883657 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558064136e50 con 0x5580641f04e0 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9d837fe700 10 client.25705 handle_command_reply: tid=0 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 client.25705 shutdown 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 msgr2=0x5580641e0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641e0530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0df780 tx=0x7f9d9c0a98e0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 shutdown_connections 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641dbdd0 0x5580641e0530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5580641f04e0 0x5580641f28d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 --2- 172.21.15.99:0/3487883657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641db4b0 0x5580641dfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 >> 172.21.15.99:0/3487883657 conn(0x558064103f40 msgr2=0x5580641cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9d837fe700 0 client.25705 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:41.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 shutdown_connections 2024-05-14T17:11:41.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 1 -- 172.21.15.99:0/3487883657 wait complete. 2024-05-14T17:11:41.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.075+0000 7f9db521cb80 20 client.25705 trim_cache size 0 max 16384 2024-05-14T17:11:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 Processor -- start 2024-05-14T17:11:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 -- start start 2024-05-14T17:11:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41b0720 con 0x5580641dbdd0 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41adf50 con 0x5580641d7350 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9db521cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41abda0 con 0x5580641db4b0 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9d9bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9da3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641ec820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:51488/0 (socket says 172.21.15.99:51488) 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53984/0 (socket says 172.21.15.99:53984) 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.097+0000 7f9da8f23700 1 -- 172.21.15.99:0/2441771605 learned_addr learned my addr 172.21.15.99:0/2441771605 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da8f23700 1 -- 172.21.15.99:0/2441771605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 msgr2=0x5580641e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da8f23700 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da8f23700 1 -- 172.21.15.99:0/2441771605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 msgr2=0x5580641ec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da8f23700 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641ec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da8f23700 1 -- 172.21.15.99:0/2441771605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580641defd0 con 0x5580641dbdd0 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da3fff700 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9da8f23700 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e2240 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0a8c80 tx=0x7f9d9c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fdc30f002d75c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9d99ffb700 1 -- 172.21.15.99:0/2441771605 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0b2170 con 0x5580641dbdd0 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9d99ffb700 1 -- 172.21.15.99:0/2441771605 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9c0a8810 con 0x5580641dbdd0 2024-05-14T17:11:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.098+0000 7f9d99ffb700 1 -- 172.21.15.99:0/2441771605 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0ad170 con 0x5580641dbdd0 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 -- 172.21.15.99:0/2441771605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 msgr2=0x5580641e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e2240 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0a8c80 tx=0x7f9d9c009480 comp rx=0 tx=0).stop 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 -- 172.21.15.99:0/2441771605 shutdown_connections 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e2240 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2441771605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 -- 172.21.15.99:0/2441771605 >> 172.21.15.99:0/2441771605 conn(0x558064103f40 msgr2=0x5580641e15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 -- 172.21.15.99:0/2441771605 shutdown_connections 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 -- 172.21.15.99:0/2441771605 wait complete. 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 -- start start 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.099+0000 7f9db521cb80 1 Processor -- start 2024-05-14T17:11:41.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41ac410 con 0x5580641dbdd0 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41ad880 con 0x5580641d7350 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41acae0 con 0x5580641db4b0 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 10 client.0 _ll_register_callbacks cb 0x5580640bfae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9d9bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53986/0 (socket says 172.21.15.99:53986) 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 -- 172.21.15.99:0/2171213060 learned_addr learned my addr 172.21.15.99:0/2171213060 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da3fff700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 -- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 msgr2=0x5580641e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 -- 172.21.15.99:0/2171213060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 msgr2=0x5580641e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 -- 172.21.15.99:0/2171213060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580641e6df0 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9d9bfff700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9da8f23700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e67b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ef00 tx=0x7f9d9c0a3cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dcc0153c1c9bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9d837fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9d837fe700 1 -- 172.21.15.99:0/2171213060 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0b2170 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9d837fe700 1 -- 172.21.15.99:0/2171213060 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f9d9c0a4f20 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.100+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9c0a8810 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9d837fe700 1 -- 172.21.15.99:0/2171213060 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c0ad750 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 10 client.16263 fetch_fsmap learned FSMap version 34 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55806425cb60 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9d837fe700 1 -- 172.21.15.99:0/2171213060 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9c0a8810 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9d837fe700 1 -- 172.21.15.99:0/2171213060 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9d9c013aa0 con 0x5580641dbdd0 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 10 client.16263 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 10 client.16263 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 20 client.16263 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55806425d120 0x55806425f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 4 client.16263 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.3, "await_for": 45.0}] 2024-05-14T17:11:41.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.3, "await_for": 45.0}) -- 0x558064130630 con 0x55806425d120 2024-05-14T17:11:41.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9da3fff700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55806425d120 0x55806425f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9d837fe700 10 client.16263 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:41.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.101+0000 7f9da3fff700 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55806425d120 0x55806425f540 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784aa03e75941133 server_cookie=322d25901b464264 in_seq=0 out_seq=0 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9d837fe700 1 -- 172.21.15.99:0/2171213060 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558064130630 con 0x55806425d120 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9d837fe700 10 client.16263 handle_command_reply: tid=0 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 client.16263 shutdown 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 msgr2=0x5580641e67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e67b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ef00 tx=0x7f9d9c0a3cf0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 shutdown_connections 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55806425d120 0x55806425f540 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580641dbdd0 0x5580641e67b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580641d7350 0x5580641e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9d837fe700 0 client.16263 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.169+0000 7f9db521cb80 1 --2- 172.21.15.99:0/2171213060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5580641db4b0 0x5580641e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 >> 172.21.15.99:0/2171213060 conn(0x558064103f40 msgr2=0x5580641e15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 shutdown_connections 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9db521cb80 1 -- 172.21.15.99:0/2171213060 wait complete. 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9db521cb80 20 client.16263 trim_cache size 0 max 16384 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 34, 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:11:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 26, 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "118b3367": { 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 26, 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 151.3, 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:11:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:11:41.172 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9d900b2aa0 msgr2=0x7f9d900b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9d900b2aa0 0x7f9d900b4f60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b2e0 tx=0x7f9d94005fd0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 msgr2=0x7f9da41a5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41a5a60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00ef30 tx=0x7f9d9c00c3a0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 shutdown_connections 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9d900b2aa0 0x7f9d900b4f60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9da41084c0 0x7f9da41a5a60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da4102440 0x7f9da419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 --2- 172.21.15.99:0/344680394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9da4102e00 0x7f9da419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 >> 172.21.15.99:0/344680394 conn(0x7f9da40793e0 msgr2=0x7f9da41003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 shutdown_connections 2024-05-14T17:11:41.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:11:41.170+0000 7f9daa986700 1 -- 172.21.15.99:0/344680394 wait complete. 2024-05-14T17:11:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:41 smithi193 ceph-mon[35962]: pgmap v250: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 754 KiB/s wr, 114 op/s 2024-05-14T17:11:41.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:41 smithi150 ceph-mon[36204]: pgmap v250: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 754 KiB/s wr, 114 op/s 2024-05-14T17:11:41.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.52 MB/sec execute 233 sec latency 366.869 ms 2024-05-14T17:11:41.833 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '118b3367', quiesce took 5.5 seconds. Will release after: 25.8 2024-05-14T17:11:42.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.49 MB/sec execute 234 sec latency 1366.950 ms 2024-05-14T17:11:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:43 smithi193 ceph-mon[35962]: pgmap v251: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 754 KiB/s wr, 114 op/s 2024-05-14T17:11:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:43 smithi150 ceph-mon[36204]: pgmap v251: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 754 KiB/s wr, 114 op/s 2024-05-14T17:11:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:43 smithi099 ceph-mon[31851]: pgmap v251: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 754 KiB/s wr, 114 op/s 2024-05-14T17:11:43.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.47 MB/sec execute 235 sec latency 2367.036 ms 2024-05-14T17:11:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 -- 172.21.15.99:0/293335648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310107940 msgr2=0x7f3310107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 --2- 172.21.15.99:0/293335648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310107940 0x7f3310107d20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3300009930 tx=0x7f330002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 -- 172.21.15.99:0/293335648 shutdown_connections 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 --2- 172.21.15.99:0/293335648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3310102340 0x7f331010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 --2- 172.21.15.99:0/293335648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3310101980 0x7f3310101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 --2- 172.21.15.99:0/293335648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310107940 0x7f3310107d20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 -- 172.21.15.99:0/293335648 >> 172.21.15.99:0/293335648 conn(0x7f3310078ad0 msgr2=0x7f3310078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 -- 172.21.15.99:0/293335648 shutdown_connections 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.997+0000 7f331679d640 1 -- 172.21.15.99:0/293335648 wait complete. 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.998+0000 7f331679d640 1 Processor -- start 2024-05-14T17:11:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.998+0000 7f331679d640 1 -- start start 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.998+0000 7f331679d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3310101980 0x7f3310198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.998+0000 7f331679d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 0x7f3310198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.998+0000 7f331679d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3310107940 0x7f331019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f331679d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3310111c00 con 0x7f3310102340 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f331679d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3310111a80 con 0x7f3310101980 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f331679d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3310111d80 con 0x7f3310107940 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 0x7f3310198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 0x7f3310198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54002/0 (socket says 172.21.15.99:54002) 2024-05-14T17:11:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 -- 172.21.15.99:0/38399063 learned_addr learned my addr 172.21.15.99:0/38399063 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330ffff640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3310101980 0x7f3310198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f3314d3a640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3310107940 0x7f331019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 -- 172.21.15.99:0/38399063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3310107940 msgr2=0x7f331019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3310107940 0x7f331019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 -- 172.21.15.99:0/38399063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3310101980 msgr2=0x7f3310198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3310101980 0x7f3310198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 -- 172.21.15.99:0/38399063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331019cfe0 con 0x7f3310102340 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330f7fe640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 0x7f3310198580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007b50 tx=0x7f32f800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330d7fa640 1 -- 172.21.15.99:0/38399063 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8007c40 con 0x7f3310102340 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331019d2d0 con 0x7f3310102340 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330d7fa640 1 -- 172.21.15.99:0/38399063 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f8007de0 con 0x7f3310102340 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f330d7fa640 1 -- 172.21.15.99:0/38399063 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800ace0 con 0x7f3310102340 2024-05-14T17:11:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:43.999+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331006a100 con 0x7f3310102340 2024-05-14T17:11:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f330d7fa640 1 -- 172.21.15.99:0/38399063 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f32f800eb30 con 0x7f3310102340 2024-05-14T17:11:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 Processor -- start 2024-05-14T17:11:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 -- start start 2024-05-14T17:11:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6dd7f0 0x55742a6ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6ecee0 0x55742a6ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55742a650c30 con 0x55742a6de110 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55742a6407d0 con 0x55742a6dd7f0 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f331958e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55742a644810 con 0x55742a6ecee0 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f330ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.002+0000 7f330ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54006/0 (socket says 172.21.15.99:54006) 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 -- 172.21.15.99:0/2640584478 learned_addr learned my addr 172.21.15.99:0/2640584478 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f3314d3a640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6dd7f0 0x55742a6ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330f7fe640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6ecee0 0x55742a6ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 -- 172.21.15.99:0/2640584478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6ecee0 msgr2=0x55742a6ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6ecee0 0x55742a6ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 -- 172.21.15.99:0/2640584478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6dd7f0 msgr2=0x55742a6ddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6dd7f0 0x55742a6ddbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 -- 172.21.15.99:0/2640584478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742a355210 con 0x55742a6de110 2024-05-14T17:11:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330d7fa640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f32fc0b2a10 0x7f32fc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f3314d3a640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6dd7f0 0x55742a6ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330f7fe640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6ecee0 0x55742a6ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f3314d3a640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f32fc0b2a10 0x7f32fc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330ffff640 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec9a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3300009930 tx=0x7f3300005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88140b8f97e2aec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f330d7fa640 1 -- 172.21.15.99:0/38399063 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f32f80e3910 con 0x7f3310102340 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f33067fc640 1 -- 172.21.15.99:0/2640584478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300040070 con 0x55742a6de110 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f33067fc640 1 -- 172.21.15.99:0/2640584478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33000316a0 con 0x55742a6de110 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.003+0000 7f33067fc640 1 -- 172.21.15.99:0/2640584478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300035040 con 0x55742a6de110 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 -- 172.21.15.99:0/2640584478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 msgr2=0x55742a6ec9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f3314d3a640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f32fc0b2a10 0x7f32fc0b4ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3308005fd0 tx=0x7f3308009600 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec9a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3300009930 tx=0x7f3300005fd0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 -- 172.21.15.99:0/2640584478 shutdown_connections 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6ecee0 0x55742a6ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec9a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 --2- 172.21.15.99:0/2640584478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6dd7f0 0x55742a6ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 -- 172.21.15.99:0/2640584478 >> 172.21.15.99:0/2640584478 conn(0x55742a6d9e00 msgr2=0x55742a6da1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 -- 172.21.15.99:0/2640584478 shutdown_connections 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 -- 172.21.15.99:0/2640584478 wait complete. 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.004+0000 7f331958e740 1 -- start start 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 Processor -- start 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55742a326790 con 0x55742a6ecee0 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55742a5fc890 con 0x55742a6de110 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55742a6efb50 con 0x55742a6dd7f0 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f331958e740 10 client.0 _ll_register_callbacks cb 0x55742a5ffd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6ea040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54012/0 (socket says 172.21.15.99:54012) 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 -- 172.21.15.99:0/1382485428 learned_addr learned my addr 172.21.15.99:0/1382485428 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 -- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 msgr2=0x55742a6ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f330ffff640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 -- 172.21.15.99:0/1382485428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 msgr2=0x55742a6e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f330f7fe640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f3314d3a640 1 -- 172.21.15.99:0/1382485428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742a421620 con 0x55742a6ecee0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.005+0000 7f330ffff640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f3314d3a640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6ea040 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3308011e70 tx=0x7f3308004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af9ae967085d51a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f32ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f32ef7fe640 1 -- 172.21.15.99:0/1382485428 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308020620 con 0x55742a6ecee0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f32ef7fe640 1 -- 172.21.15.99:0/1382485428 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f33080207c0 con 0x55742a6ecee0 2024-05-14T17:11:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33000316a0 con 0x55742a6ecee0 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f32ef7fe640 1 -- 172.21.15.99:0/1382485428 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33080209c0 con 0x55742a6ecee0 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 10 client.16275 fetch_fsmap learned FSMap version 34 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55742a6fb630 con 0x55742a6ecee0 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f32ef7fe640 1 -- 172.21.15.99:0/1382485428 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33000316a0 con 0x55742a6ecee0 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f32ef7fe640 1 -- 172.21.15.99:0/1382485428 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f330800f040 con 0x55742a6ecee0 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 10 client.16275 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 20 client.16275 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a6fcb40 0x55742a6fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 4 client.16275 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.006+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55742a652b80 con 0x55742a6fcb40 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.007+0000 7f330ffff640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a6fcb40 0x55742a6fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.007+0000 7f32ef7fe640 10 client.16275 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.007+0000 7f330ffff640 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a6fcb40 0x55742a6fef50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f938eb342adce7d server_cookie=904433ca9a39f78f in_seq=0 out_seq=0 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f32ef7fe640 1 -- 172.21.15.99:0/1382485428 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55742a652b80 con 0x55742a6fcb40 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f32ef7fe640 10 client.16275 handle_command_reply: tid=0 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 client.16275 shutdown 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 msgr2=0x55742a6ea040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6ea040 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3308011e70 tx=0x7f3308004a40 comp rx=0 tx=0).stop 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 shutdown_connections 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a6fcb40 0x55742a6fef50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6ea040 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f32ef7fe640 0 client.16275 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 --2- 172.21.15.99:0/1382485428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 >> 172.21.15.99:0/1382485428 conn(0x55742a6d9e00 msgr2=0x55742a6dfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 shutdown_connections 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 1 -- 172.21.15.99:0/1382485428 wait complete. 2024-05-14T17:11:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.008+0000 7f331958e740 20 client.16275 trim_cache size 0 max 16384 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.023+0000 7f331958e740 1 Processor -- start 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.023+0000 7f331958e740 1 -- start start 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f331958e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6d8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f331958e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f331958e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6ecee0 0x55742a704810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f331958e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33101aec40 con 0x55742a6de110 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f331958e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33101ac4b0 con 0x55742a6ecee0 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f331958e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33101aa300 con 0x55742a6dd7f0 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54018/0 (socket says 172.21.15.99:54018) 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 -- 172.21.15.99:0/3646051843 learned_addr learned my addr 172.21.15.99:0/3646051843 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330ffff640 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6d8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f3314d3a640 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6ecee0 0x55742a704810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 -- 172.21.15.99:0/3646051843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 msgr2=0x55742a6d8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6d8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 -- 172.21.15.99:0/3646051843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6ecee0 msgr2=0x55742a704810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6ecee0 0x55742a704810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 -- 172.21.15.99:0/3646051843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742a6d9c30 con 0x55742a6de110 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f3314d3a640 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6ecee0 0x55742a704810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f330f7fe640 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec340 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f32f80aa9b0 tx=0x7f32f800a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cbf17ceb8937aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.024+0000 7f3305ffb640 1 -- 172.21.15.99:0/3646051843 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80aa420 con 0x55742a6de110 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f3305ffb640 1 -- 172.21.15.99:0/3646051843 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f80b33c0 con 0x55742a6de110 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f3305ffb640 1 -- 172.21.15.99:0/3646051843 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80af060 con 0x55742a6de110 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 -- 172.21.15.99:0/3646051843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 msgr2=0x55742a6ec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec340 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f32f80aa9b0 tx=0x7f32f800a700 comp rx=0 tx=0).stop 2024-05-14T17:11:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 -- 172.21.15.99:0/3646051843 shutdown_connections 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6ecee0 0x55742a704810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6de110 0x55742a6ec340 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 --2- 172.21.15.99:0/3646051843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6d8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 -- 172.21.15.99:0/3646051843 >> 172.21.15.99:0/3646051843 conn(0x55742a6d9e00 msgr2=0x55742a6d8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 -- 172.21.15.99:0/3646051843 shutdown_connections 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.025+0000 7f331958e740 1 -- 172.21.15.99:0/3646051843 wait complete. 2024-05-14T17:11:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 -- start start 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 Processor -- start 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33101aa970 con 0x55742a6ecee0 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33101abde0 con 0x55742a6de110 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33101ab040 con 0x55742a6dd7f0 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f331958e740 10 client.0 _ll_register_callbacks cb 0x7f3308011af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f3314d3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f3314d3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54026/0 (socket says 172.21.15.99:54026) 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f3314d3a640 1 -- 172.21.15.99:0/2032347563 learned_addr learned my addr 172.21.15.99:0/2032347563 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f330f7fe640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.026+0000 7f330ffff640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f3314d3a640 1 -- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 msgr2=0x55742a6f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f3314d3a640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6f09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f3314d3a640 1 -- 172.21.15.99:0/2032347563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 msgr2=0x55742a6f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f3314d3a640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6f0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f3314d3a640 1 -- 172.21.15.99:0/2032347563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742a766440 con 0x55742a6ecee0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f330ffff640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f3314d3a640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6f1440 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3308018040 tx=0x7f3308004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc99bb89afa86fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f32ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f32ef7fe640 1 -- 172.21.15.99:0/2032347563 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330803f060 con 0x55742a6ecee0 2024-05-14T17:11:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f32ef7fe640 1 -- 172.21.15.99:0/2032347563 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f3308020420 con 0x55742a6ecee0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55742a64c360 con 0x55742a6ecee0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f32ef7fe640 1 -- 172.21.15.99:0/2032347563 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f330801fc10 con 0x55742a6ecee0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 10 client.16281 fetch_fsmap learned FSMap version 34 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55742a767730 con 0x55742a6ecee0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f32ef7fe640 1 -- 172.21.15.99:0/2032347563 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55742a64c360 con 0x55742a6ecee0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f32ef7fe640 1 -- 172.21.15.99:0/2032347563 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f330800f050 con 0x55742a6ecee0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 10 client.16281 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 20 client.16281 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.027+0000 7f331958e740 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a767e50 0x55742a76a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.028+0000 7f331958e740 4 client.16281 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.028+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55742a5c65f0 con 0x55742a767e50 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.028+0000 7f330ffff640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a767e50 0x55742a76a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.028+0000 7f32ef7fe640 10 client.16281 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.028+0000 7f330ffff640 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a767e50 0x55742a76a270 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2eb52bca3633f2 server_cookie=11e27ccb115447ad in_seq=0 out_seq=0 2024-05-14T17:11:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f32ef7fe640 1 -- 172.21.15.99:0/2032347563 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55742a5c65f0 con 0x55742a767e50 2024-05-14T17:11:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f32ef7fe640 10 client.16281 handle_command_reply: tid=0 2024-05-14T17:11:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 client.16281 shutdown 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 msgr2=0x55742a6f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6f1440 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3308018040 tx=0x7f3308004d60 comp rx=0 tx=0).stop 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 shutdown_connections 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55742a767e50 0x55742a76a270 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742a6ecee0 0x55742a6f1440 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55742a6de110 0x55742a6f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 --2- 172.21.15.99:0/2032347563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742a6dd7f0 0x55742a6f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f32ef7fe640 0 client.16281 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 >> 172.21.15.99:0/2032347563 conn(0x55742a6d9e00 msgr2=0x55742a6dfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 shutdown_connections 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.069+0000 7f331958e740 1 -- 172.21.15.99:0/2032347563 wait complete. 2024-05-14T17:11:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331958e740 20 client.16281 trim_cache size 0 max 16384 2024-05-14T17:11:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:11:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:11:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:11:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f32fc0b2a10 msgr2=0x7f32fc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f32fc0b2a10 0x7f32fc0b4ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3308005fd0 tx=0x7f3308009600 comp rx=0 tx=0).stop 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 msgr2=0x7f3310198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 0x7f3310198580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007b50 tx=0x7f32f800e420 comp rx=0 tx=0).stop 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 shutdown_connections 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f32fc0b2a10 0x7f32fc0b4ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3310107940 0x7f331019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3310102340 0x7f3310198580 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 --2- 172.21.15.99:0/38399063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3310101980 0x7f3310198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 >> 172.21.15.99:0/38399063 conn(0x7f3310078ad0 msgr2=0x7f33101044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 shutdown_connections 2024-05-14T17:11:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:44.070+0000 7f331679d640 1 -- 172.21.15.99:0/38399063 wait complete. 2024-05-14T17:11:44.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.45 MB/sec execute 236 sec latency 3367.134 ms 2024-05-14T17:11:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:45 smithi193 ceph-mon[35962]: pgmap v252: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-14T17:11:45.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:45 smithi150 ceph-mon[36204]: pgmap v252: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-14T17:11:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:45 smithi099 ceph-mon[31851]: pgmap v252: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-05-14T17:11:45.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.43 MB/sec execute 237 sec latency 4367.219 ms 2024-05-14T17:11:46.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.40 MB/sec execute 238 sec latency 5367.312 ms 2024-05-14T17:11:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:47 smithi193 ceph-mon[35962]: pgmap v253: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 918 KiB/s wr, 81 op/s 2024-05-14T17:11:47.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:47 smithi150 ceph-mon[36204]: pgmap v253: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 918 KiB/s wr, 81 op/s 2024-05-14T17:11:47.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:47 smithi099 ceph-mon[31851]: pgmap v253: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 918 KiB/s wr, 81 op/s 2024-05-14T17:11:47.538 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.38 MB/sec execute 239 sec latency 6367.406 ms 2024-05-14T17:11:48.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.36 MB/sec execute 240 sec latency 7367.516 ms 2024-05-14T17:11:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:49 smithi193 ceph-mon[35962]: pgmap v254: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 918 KiB/s wr, 81 op/s 2024-05-14T17:11:49.316 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:49 smithi099 ceph-mon[31851]: pgmap v254: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 918 KiB/s wr, 81 op/s 2024-05-14T17:11:49.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:49 smithi150 ceph-mon[36204]: pgmap v254: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 918 KiB/s wr, 81 op/s 2024-05-14T17:11:49.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.34 MB/sec execute 241 sec latency 8367.585 ms 2024-05-14T17:11:50.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.31 MB/sec execute 242 sec latency 9367.689 ms 2024-05-14T17:11:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:51 smithi193 ceph-mon[35962]: pgmap v255: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-05-14T17:11:51.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:51 smithi150 ceph-mon[36204]: pgmap v255: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-05-14T17:11:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:51 smithi099 ceph-mon[31851]: pgmap v255: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-05-14T17:11:51.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.29 MB/sec execute 243 sec latency 10367.773 ms 2024-05-14T17:11:52.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.27 MB/sec execute 244 sec latency 11367.883 ms 2024-05-14T17:11:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:53 smithi193 ceph-mon[35962]: pgmap v256: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 900 KiB/s wr, 71 op/s 2024-05-14T17:11:53.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:53 smithi150 ceph-mon[36204]: pgmap v256: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 900 KiB/s wr, 71 op/s 2024-05-14T17:11:53.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:53 smithi099 ceph-mon[31851]: pgmap v256: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 900 KiB/s wr, 71 op/s 2024-05-14T17:11:53.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.25 MB/sec execute 245 sec latency 12367.935 ms 2024-05-14T17:11:54.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.23 MB/sec execute 246 sec latency 13368.019 ms 2024-05-14T17:11:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:55 smithi193 ceph-mon[35962]: pgmap v257: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.1 MiB/s wr, 80 op/s 2024-05-14T17:11:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:55 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:55.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:55 smithi150 ceph-mon[36204]: pgmap v257: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.1 MiB/s wr, 80 op/s 2024-05-14T17:11:55.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:55 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:55.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:55 smithi099 ceph-mon[31851]: pgmap v257: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.1 MiB/s wr, 80 op/s 2024-05-14T17:11:55.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:55 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:55.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.21 MB/sec execute 247 sec latency 14368.112 ms 2024-05-14T17:11:56.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:56 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:56.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:56 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:56.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:56 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:11:56.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:56 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:56.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:56 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:56.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:56 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:11:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:56 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:56 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:56.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:56 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:11:56.539 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.18 MB/sec execute 248 sec latency 15368.225 ms 2024-05-14T17:11:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 -- 172.21.15.99:0/989717628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 msgr2=0x7f807c109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 --2- 172.21.15.99:0/989717628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c109030 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f806c009ca0 tx=0x7f806c02a950 comp rx=0 tx=0).stop 2024-05-14T17:11:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 -- 172.21.15.99:0/989717628 shutdown_connections 2024-05-14T17:11:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 --2- 172.21.15.99:0/989717628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 0x7f807c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 --2- 172.21.15.99:0/989717628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f807c06b280 0x7f807c06b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 --2- 172.21.15.99:0/989717628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c109030 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 -- 172.21.15.99:0/989717628 >> 172.21.15.99:0/989717628 conn(0x7f807c077750 msgr2=0x7f807c077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 -- 172.21.15.99:0/989717628 shutdown_connections 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.169+0000 7f8080a93640 1 -- 172.21.15.99:0/989717628 wait complete. 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 Processor -- start 2024-05-14T17:11:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 -- start start 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f807c06b280 0x7f807c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 0x7f807c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f807c112da0 con 0x7f807c108c50 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f807c112c20 con 0x7f807c06bc40 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f8080a93640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f807c112f20 con 0x7f807c06b280 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f807ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.170+0000 7f807a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f807c06b280 0x7f807c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35828/0 (socket says 172.21.15.99:35828) 2024-05-14T17:11:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 -- 172.21.15.99:0/1306893119 learned_addr learned my addr 172.21.15.99:0/1306893119 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f8079d9b640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 0x7f807c199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 -- 172.21.15.99:0/1306893119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f807c06b280 msgr2=0x7f807c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f807c06b280 0x7f807c199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 -- 172.21.15.99:0/1306893119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 msgr2=0x7f807c199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 0x7f807c199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 -- 172.21.15.99:0/1306893119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c19e310 con 0x7f807c108c50 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f8079d9b640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 0x7f807c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f807ad9d640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c19dbe0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8074009fd0 tx=0x7f807400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f806b7fe640 1 -- 172.21.15.99:0/1306893119 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074011830 con 0x7f807c108c50 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f806b7fe640 1 -- 172.21.15.99:0/1306893119 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8074016070 con 0x7f807c108c50 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c19e600 con 0x7f807c108c50 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f806b7fe640 1 -- 172.21.15.99:0/1306893119 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074010de0 con 0x7f807c108c50 2024-05-14T17:11:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.171+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1a5e90 con 0x7f807c108c50 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f806b7fe640 1 -- 172.21.15.99:0/1306893119 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f80740119d0 con 0x7f807c108c50 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 Processor -- start 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 -- start start 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761c0210 0x5605761ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 0x5605761ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560576132c30 con 0x5605761caad0 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5605761227d0 con 0x5605761c0210 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.174+0000 7f80837ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560576126810 con 0x5605761bf8f0 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 0x5605761ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f807a59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761c0210 0x5605761ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 0x5605761ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35842/0 (socket says 172.21.15.99:35842) 2024-05-14T17:11:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f807a59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761c0210 0x5605761ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:41012/0 (socket says 172.21.15.99:41012) 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 -- 172.21.15.99:0/3137012774 learned_addr learned my addr 172.21.15.99:0/3137012774 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f806b7fe640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80640b2a10 0x7f80640b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 -- 172.21.15.99:0/3137012774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 msgr2=0x5605761bfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 -- 172.21.15.99:0/3137012774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761c0210 msgr2=0x5605761ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761c0210 0x5605761ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 -- 172.21.15.99:0/3137012774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560575e37210 con 0x5605761caad0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f807ad9d640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80640b2a10 0x7f80640b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8079d9b640 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 0x5605761ccec0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8070016560 tx=0x7f807000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c0e09e44b58994 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f806b7fe640 1 -- 172.21.15.99:0/1306893119 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f80740e1bb0 con 0x7f807c108c50 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8068ff9640 1 -- 172.21.15.99:0/3137012774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070018430 con 0x5605761caad0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8068ff9640 1 -- 172.21.15.99:0/3137012774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8070018a80 con 0x5605761caad0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.175+0000 7f8068ff9640 1 -- 172.21.15.99:0/3137012774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700177d0 con 0x5605761caad0 2024-05-14T17:11:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 -- 172.21.15.99:0/3137012774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 msgr2=0x5605761ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f807ad9d640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80640b2a10 0x7f80640b4ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f807400bde0 tx=0x7f8074011000 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 0x5605761ccec0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8070016560 tx=0x7f807000a340 comp rx=0 tx=0).stop 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 -- 172.21.15.99:0/3137012774 shutdown_connections 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761caad0 0x5605761ccec0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761c0210 0x5605761ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 --2- 172.21.15.99:0/3137012774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 -- 172.21.15.99:0/3137012774 >> 172.21.15.99:0/3137012774 conn(0x5605761bbe00 msgr2=0x5605761bc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 -- 172.21.15.99:0/3137012774 shutdown_connections 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.176+0000 7f80837ac740 1 -- 172.21.15.99:0/3137012774 wait complete. 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 -- start start 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 Processor -- start 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560575e08790 con 0x5605761c0210 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5605760de890 con 0x5605761caad0 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605761cd740 con 0x5605761bf8f0 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f80837ac740 10 client.0 _ll_register_callbacks cb 0x5605760e1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35856/0 (socket says 172.21.15.99:35856) 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 -- 172.21.15.99:0/2683775691 learned_addr learned my addr 172.21.15.99:0/2683775691 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 -- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 msgr2=0x5605761ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 -- 172.21.15.99:0/2683775691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 msgr2=0x5605761c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.177+0000 7f807a59c640 1 -- 172.21.15.99:0/2683775691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560575f03620 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f807a59c640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761c76e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f806c009ca0 tx=0x7f806c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0af5b150e4f799e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f8061ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f8061ffb640 1 -- 172.21.15.99:0/2683775691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c049060 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f8061ffb640 1 -- 172.21.15.99:0/2683775691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f806c003d80 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8070018a80 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f8061ffb640 1 -- 172.21.15.99:0/2683775691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f806c034040 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 10 client.16293 fetch_fsmap learned FSMap version 34 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605761d9150 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f8061ffb640 1 -- 172.21.15.99:0/2683775691 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8070018a80 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f8061ffb640 1 -- 172.21.15.99:0/2683775691 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f806c030560 con 0x5605761c0210 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 10 client.16293 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:11:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 20 client.16293 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5605761da610 0x5605761dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 4 client.16293 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.178+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560576134b80 con 0x5605761da610 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.179+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5605761da610 0x5605761dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.179+0000 7f8061ffb640 10 client.16293 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.179+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5605761da610 0x5605761dca20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ecda779844ad3e server_cookie=e7d08a0638ecb7a7 in_seq=0 out_seq=0 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f8061ffb640 1 -- 172.21.15.99:0/2683775691 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560576134b80 con 0x5605761da610 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f8061ffb640 10 client.16293 handle_command_reply: tid=0 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 client.16293 shutdown 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 msgr2=0x5605761c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761c76e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f806c009ca0 tx=0x7f806c02f5d0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 shutdown_connections 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5605761da610 0x5605761dca20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761c76e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f8061ffb640 0 client.16293 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 --2- 172.21.15.99:0/2683775691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 >> 172.21.15.99:0/2683775691 conn(0x5605761bbe00 msgr2=0x5605761bb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 shutdown_connections 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 1 -- 172.21.15.99:0/2683775691 wait complete. 2024-05-14T17:11:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.180+0000 7f80837ac740 20 client.16293 trim_cache size 0 max 16384 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 Processor -- start 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 -- start start 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1b0fc0 con 0x5605761c0210 2024-05-14T17:11:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1ae830 con 0x5605761caad0 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.195+0000 7f80837ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1ac680 con 0x5605761bf8f0 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35862/0 (socket says 172.21.15.99:35862) 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 -- 172.21.15.99:0/3071840002 learned_addr learned my addr 172.21.15.99:0/3071840002 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 -- 172.21.15.99:0/3071840002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 msgr2=0x5605761bb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f8079d9b640 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 -- 172.21.15.99:0/3071840002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 msgr2=0x5605761e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 -- 172.21.15.99:0/3071840002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605761b6510 con 0x5605761c0210 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f8079d9b640 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f807a59c640 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d6610 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f806c009bf0 tx=0x7f806c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e0d922781dbf51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f8063fff640 1 -- 172.21.15.99:0/3071840002 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c049070 con 0x5605761c0210 2024-05-14T17:11:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f8063fff640 1 -- 172.21.15.99:0/3071840002 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806c030b50 con 0x5605761c0210 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.196+0000 7f8063fff640 1 -- 172.21.15.99:0/3071840002 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c04ecc0 con 0x5605761c0210 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- 172.21.15.99:0/3071840002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 msgr2=0x5605761d6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d6610 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f806c009bf0 tx=0x7f806c005c80 comp rx=0 tx=0).stop 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- 172.21.15.99:0/3071840002 shutdown_connections 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761caad0 0x5605761e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d6610 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- 172.21.15.99:0/3071840002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761bf8f0 0x5605761bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- 172.21.15.99:0/3071840002 >> 172.21.15.99:0/3071840002 conn(0x5605761bbe00 msgr2=0x5605761bb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- 172.21.15.99:0/3071840002 shutdown_connections 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- 172.21.15.99:0/3071840002 wait complete. 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- start start 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 Processor -- start 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761bf8f0 0x5605761d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761caad0 0x560576243f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1accf0 con 0x5605761c0210 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1ae160 con 0x5605761bf8f0 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f807c1ad3c0 con 0x5605761caad0 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f80837ac740 10 client.0 _ll_register_callbacks cb 0x7f806c02fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.197+0000 7f807a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35870/0 (socket says 172.21.15.99:35870) 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 -- 172.21.15.99:0/2000576170 learned_addr learned my addr 172.21.15.99:0/2000576170 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761bf8f0 0x5605761d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761caad0 0x560576243f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 -- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761caad0 msgr2=0x560576243f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761caad0 0x560576243f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 -- 172.21.15.99:0/2000576170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761bf8f0 msgr2=0x5605761d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761bf8f0 0x5605761d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 -- 172.21.15.99:0/2000576170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560576244440 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f807a59c640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d10d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f806c002230 tx=0x7f806c030440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4904ed46a0ffb45 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8061ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8061ffb640 1 -- 172.21.15.99:0/2000576170 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c05b020 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8061ffb640 1 -- 172.21.15.99:0/2000576170 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f806c04ec40 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806c030b50 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8061ffb640 1 -- 172.21.15.99:0/2000576170 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f806c049050 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f80837ac740 10 client.16299 fetch_fsmap learned FSMap version 34 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605762455c0 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.198+0000 7f8061ffb640 1 -- 172.21.15.99:0/2000576170 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806c030b50 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f8061ffb640 1 -- 172.21.15.99:0/2000576170 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f806c0606a0 con 0x5605761c0210 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f80837ac740 10 client.16299 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f80837ac740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:11:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f80837ac740 20 client.16299 populate_metadata read hostname 'smithi099' 2024-05-14T17:11:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f80837ac740 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560576245bc0 0x560576247fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f80837ac740 4 client.16299 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56057612e360 con 0x560576245bc0 2024-05-14T17:11:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560576245bc0 0x560576247fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f8061ffb640 10 client.16299 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.199+0000 7f8079d9b640 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560576245bc0 0x560576247fb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9640f6fcad2d41cd server_cookie=c048a5d441dec101 in_seq=0 out_seq=0 2024-05-14T17:11:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f8061ffb640 1 -- 172.21.15.99:0/2000576170 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56057612e360 con 0x560576245bc0 2024-05-14T17:11:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f8061ffb640 10 client.16299 handle_command_reply: tid=0 2024-05-14T17:11:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 client.16299 shutdown 2024-05-14T17:11:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 msgr2=0x5605761d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d10d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f806c002230 tx=0x7f806c030440 comp rx=0 tx=0).stop 2024-05-14T17:11:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 shutdown_connections 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560576245bc0 0x560576247fb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605761caad0 0x560576243f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605761c0210 0x5605761d10d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f8061ffb640 0 client.16299 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 --2- 172.21.15.99:0/2000576170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605761bf8f0 0x5605761d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 >> 172.21.15.99:0/2000576170 conn(0x5605761bbe00 msgr2=0x5605761d5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 shutdown_connections 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.223+0000 7f80837ac740 1 -- 172.21.15.99:0/2000576170 wait complete. 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f80837ac740 20 client.16299 trim_cache size 0 max 16384 2024-05-14T17:11:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:11:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:11:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:11:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80640b2a10 msgr2=0x7f80640b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80640b2a10 0x7f80640b4ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f807400bde0 tx=0x7f8074011000 comp rx=0 tx=0).stop 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 msgr2=0x7f807c19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c19dbe0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8074009fd0 tx=0x7f807400b8e0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 shutdown_connections 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f80640b2a10 0x7f80640b4ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f807c108c50 0x7f807c19dbe0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f807c06bc40 0x7f807c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 --2- 172.21.15.99:0/1306893119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f807c06b280 0x7f807c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 >> 172.21.15.99:0/1306893119 conn(0x7f807c077750 msgr2=0x7f807c100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 shutdown_connections 2024-05-14T17:11:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:11:57.224+0000 7f8080a93640 1 -- 172.21.15.99:0/1306893119 wait complete. 2024-05-14T17:11:57.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.16 MB/sec execute 249 sec latency 16368.310 ms 2024-05-14T17:11:57.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:57 smithi150 ceph-mon[36204]: pgmap v258: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 520 KiB/s wr, 35 op/s 2024-05-14T17:11:57.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:57 smithi099 ceph-mon[31851]: pgmap v258: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 520 KiB/s wr, 35 op/s 2024-05-14T17:11:57.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:57 smithi193 ceph-mon[35962]: pgmap v258: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 520 KiB/s wr, 35 op/s 2024-05-14T17:11:58.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.14 MB/sec execute 250 sec latency 17368.429 ms 2024-05-14T17:11:59.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.12 MB/sec execute 251 sec latency 18368.535 ms 2024-05-14T17:11:59.623 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:11:59 smithi099 ceph-mon[31851]: pgmap v259: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 520 KiB/s wr, 35 op/s 2024-05-14T17:11:59.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:11:59 smithi150 ceph-mon[36204]: pgmap v259: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 520 KiB/s wr, 35 op/s 2024-05-14T17:11:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:11:59 smithi193 ceph-mon[35962]: pgmap v259: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 520 KiB/s wr, 35 op/s 2024-05-14T17:12:00.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.10 MB/sec execute 252 sec latency 19368.659 ms 2024-05-14T17:12:01.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.08 MB/sec execute 253 sec latency 20368.775 ms 2024-05-14T17:12:01.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:01 smithi150 ceph-mon[36204]: pgmap v260: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 542 KiB/s wr, 43 op/s 2024-05-14T17:12:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:01 smithi099 ceph-mon[31851]: pgmap v260: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 542 KiB/s wr, 43 op/s 2024-05-14T17:12:01.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:01 smithi193 ceph-mon[35962]: pgmap v260: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 542 KiB/s wr, 43 op/s 2024-05-14T17:12:02.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.06 MB/sec execute 254 sec latency 21368.842 ms 2024-05-14T17:12:03.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.04 MB/sec execute 255 sec latency 22368.967 ms 2024-05-14T17:12:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:03 smithi150 ceph-mon[36204]: pgmap v261: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 258 KiB/s wr, 17 op/s 2024-05-14T17:12:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:03 smithi099 ceph-mon[31851]: pgmap v261: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 258 KiB/s wr, 17 op/s 2024-05-14T17:12:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:03 smithi193 ceph-mon[35962]: pgmap v261: 97 pgs: 97 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 258 KiB/s wr, 17 op/s 2024-05-14T17:12:04.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.02 MB/sec execute 256 sec latency 23369.087 ms 2024-05-14T17:12:05.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 5.00 MB/sec execute 257 sec latency 24369.180 ms 2024-05-14T17:12:05.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:05 smithi150 ceph-mon[36204]: pgmap v262: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 362 KiB/s wr, 41 op/s 2024-05-14T17:12:05.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:05 smithi099 ceph-mon[31851]: pgmap v262: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 362 KiB/s wr, 41 op/s 2024-05-14T17:12:05.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:05 smithi193 ceph-mon[35962]: pgmap v262: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 362 KiB/s wr, 41 op/s 2024-05-14T17:12:06.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 4.98 MB/sec execute 258 sec latency 25369.230 ms 2024-05-14T17:12:06.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:06 smithi150 ceph-mon[36204]: pgmap v263: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 127 KiB/s wr, 32 op/s 2024-05-14T17:12:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:06 smithi099 ceph-mon[31851]: pgmap v263: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 127 KiB/s wr, 32 op/s 2024-05-14T17:12:06.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:06 smithi193 ceph-mon[35962]: pgmap v263: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 127 KiB/s wr, 32 op/s 2024-05-14T17:12:07.540 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 4.96 MB/sec execute 259 sec latency 26369.290 ms 2024-05-14T17:12:07.633 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:12:07.890 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:12:08.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 4.95 MB/sec execute 260 sec latency 27369.379 ms 2024-05-14T17:12:09.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:08 smithi099 ceph-mon[31851]: pgmap v264: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 127 KiB/s wr, 32 op/s 2024-05-14T17:12:09.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:08 smithi193 ceph-mon[35962]: pgmap v264: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 127 KiB/s wr, 32 op/s 2024-05-14T17:12:09.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:08 smithi150 ceph-mon[36204]: pgmap v264: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 127 KiB/s wr, 32 op/s 2024-05-14T17:12:09.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 4.93 MB/sec execute 261 sec latency 28369.431 ms 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 -- 172.21.15.99:0/844917501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 msgr2=0x7f711c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 --2- 172.21.15.99:0/844917501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 0x7f711c06c190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f71100099c0 tx=0x7f7110009cd0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 -- 172.21.15.99:0/844917501 shutdown_connections 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 --2- 172.21.15.99:0/844917501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711c06c6d0 0x7f711c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 --2- 172.21.15.99:0/844917501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c109630 0x7f711c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 --2- 172.21.15.99:0/844917501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 0x7f711c06c190 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 -- 172.21.15.99:0/844917501 >> 172.21.15.99:0/844917501 conn(0x7f711c0794e0 msgr2=0x7f711c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:09.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 -- 172.21.15.99:0/844917501 shutdown_connections 2024-05-14T17:12:09.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.568+0000 7f71213d0700 1 -- 172.21.15.99:0/844917501 wait complete. 2024-05-14T17:12:09.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.569+0000 7f71213d0700 1 Processor -- start 2024-05-14T17:12:09.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.569+0000 7f71213d0700 1 -- start start 2024-05-14T17:12:09.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f71213d0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 0x7f711c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:09.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f71213d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 0x7f711c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:09.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f71213d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711c109630 0x7f711c10e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f71213d0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f711c113740 con 0x7f711c06bd10 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f71213d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1135c0 con 0x7f711c06c6d0 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f71213d0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1138c0 con 0x7f711c109630 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 0x7f711c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 0x7f711c1a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:45586/0 (socket says 172.21.15.99:45586) 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 -- 172.21.15.99:0/3851083930 learned_addr learned my addr 172.21.15.99:0/3851083930 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711b7fe700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711c109630 0x7f711c10e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 -- 172.21.15.99:0/3851083930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711c109630 msgr2=0x7f711c10e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711c109630 0x7f711c10e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 -- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 msgr2=0x7f711c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:09.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 0x7f711c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.570+0000 7f711a7fc700 1 -- 172.21.15.99:0/3851083930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c10ec40 con 0x7f711c06c6d0 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.571+0000 7f711a7fc700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 0x7f711c1a0d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7110009fd0 tx=0x7f7110005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.571+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110003780 con 0x7f711c06c6d0 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.571+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110004ce0 con 0x7f711c06c6d0 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.571+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110003dd0 con 0x7f711c06c6d0 2024-05-14T17:12:09.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.571+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c10eed0 con 0x7f711c06c6d0 2024-05-14T17:12:09.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.571+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c19c220 con 0x7f711c06c6d0 2024-05-14T17:12:09.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.574+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7108005330 con 0x7f711c06c6d0 2024-05-14T17:12:09.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.575+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7110029060 con 0x7f711c06c6d0 2024-05-14T17:12:09.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.576+0000 7f7103fff700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f71040b2c10 0x7f71040b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:09.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.576+0000 7f711affd700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f71040b2c10 0x7f71040b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:09.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.576+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f71100e2f20 con 0x7f711c06c6d0 2024-05-14T17:12:09.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.576+0000 7f711affd700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f71040b2c10 0x7f71040b50d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f710c005e90 tx=0x7f710c005d00 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:09.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.581+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7110013070 con 0x7f711c06c6d0 2024-05-14T17:12:09.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.836+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71080058f0 con 0x7f711c06c6d0 2024-05-14T17:12:09.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.837+0000 7f7103fff700 1 -- 172.21.15.99:0/3851083930 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11876 (secure 0 0 0) 0x7f71100aa9c0 con 0x7f711c06c6d0 2024-05-14T17:12:09.837 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:12:09.837 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:11:35.000334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:12:09.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f71040b2c10 msgr2=0x7f71040b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f71040b2c10 0x7f71040b50d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f710c005e90 tx=0x7f710c005d00 comp rx=0 tx=0).stop 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 msgr2=0x7f711c1a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 0x7f711c1a0d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7110009fd0 tx=0x7f7110005f10 comp rx=0 tx=0).stop 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 shutdown_connections 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f71040b2c10 0x7f71040b50d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711c109630 0x7f711c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f711c06bd10 0x7f711c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 --2- 172.21.15.99:0/3851083930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f711c06c6d0 0x7f711c1a0d10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 >> 172.21.15.99:0/3851083930 conn(0x7f711c0794e0 msgr2=0x7f711c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 shutdown_connections 2024-05-14T17:12:09.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:09.840+0000 7f71213d0700 1 -- 172.21.15.99:0/3851083930 wait complete. 2024-05-14T17:12:09.841 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 34 2024-05-14T17:12:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 -- 172.21.15.99:0/1357733576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988101980 msgr2=0x7f5988101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 --2- 172.21.15.99:0/1357733576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988101980 0x7f5988101e00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f597c009930 tx=0x7f597c02a420 comp rx=0 tx=0).stop 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 -- 172.21.15.99:0/1357733576 shutdown_connections 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 --2- 172.21.15.99:0/1357733576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988102340 0x7f598810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 --2- 172.21.15.99:0/1357733576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988101980 0x7f5988101e00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 --2- 172.21.15.99:0/1357733576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988107940 0x7f5988107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 -- 172.21.15.99:0/1357733576 >> 172.21.15.99:0/1357733576 conn(0x7f5988078ad0 msgr2=0x7f5988078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 -- 172.21.15.99:0/1357733576 shutdown_connections 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 -- 172.21.15.99:0/1357733576 wait complete. 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 Processor -- start 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.314+0000 7f598e57f640 1 -- start start 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598e57f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988101980 0x7f59881992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598e57f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 0x7f5988199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598e57f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988107940 0x7f598819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598e57f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5988111c20 con 0x7f5988102340 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598e57f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5988111aa0 con 0x7f5988107940 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598e57f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5988111da0 con 0x7f5988101980 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 0x7f5988199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 0x7f5988199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45572/0 (socket says 172.21.15.99:45572) 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 -- 172.21.15.99:0/3944575466 learned_addr learned my addr 172.21.15.99:0/3944575466 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988107940 0x7f598819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f5987fff640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988101980 0x7f59881992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 -- 172.21.15.99:0/3944575466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988101980 msgr2=0x7f59881992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988101980 0x7f59881992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 -- 172.21.15.99:0/3944575466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988107940 msgr2=0x7f598819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988107940 0x7f598819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 -- 172.21.15.99:0/3944575466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598819e2d0 con 0x7f5988102340 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f5987fff640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988101980 0x7f59881992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988107940 0x7f598819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.315+0000 7f59877fe640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 0x7f5988199820 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f597c004770 tx=0x7f597c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.316+0000 7f59857fa640 1 -- 172.21.15.99:0/3944575466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c03f070 con 0x7f5988102340 2024-05-14T17:12:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.316+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598819e560 con 0x7f5988102340 2024-05-14T17:12:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.316+0000 7f59857fa640 1 -- 172.21.15.99:0/3944575466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f597c03b460 con 0x7f5988102340 2024-05-14T17:12:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.316+0000 7f59857fa640 1 -- 172.21.15.99:0/3944575466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c035070 con 0x7f5988102340 2024-05-14T17:12:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.316+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59881a5e50 con 0x7f5988102340 2024-05-14T17:12:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 Processor -- start 2024-05-14T17:12:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f59857fa640 1 -- 172.21.15.99:0/3944575466 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f597c03b8d0 con 0x7f5988102340 2024-05-14T17:12:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 -- start start 2024-05-14T17:12:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564f8d619c30 con 0x564f8d6a7130 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564f8d6097d0 con 0x564f8d6b5ee0 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.318+0000 7f5991373740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f8d60d810 con 0x564f8d6a6810 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f598cb1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45586/0 (socket says 172.21.15.99:45586) 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f598cb1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6a6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:51096/0 (socket says 172.21.15.99:51096) 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 -- 172.21.15.99:0/2802790484 learned_addr learned my addr 172.21.15.99:0/2802790484 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f59877fe640 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 -- 172.21.15.99:0/2802790484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 msgr2=0x564f8d6a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 -- 172.21.15.99:0/2802790484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 msgr2=0x564f8d6b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 -- 172.21.15.99:0/2802790484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8d31e210 con 0x564f8d6a7130 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f59857fa640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f59700b2a10 0x7f59700b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f59700b2a10 0x7f59700b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f5987fff640 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b59a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5978009c50 tx=0x7f597800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed450357e6acc4f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f59857fa640 1 -- 172.21.15.99:0/3944575466 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f597c102a10 con 0x7f5988102340 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f596e7fc640 1 -- 172.21.15.99:0/2802790484 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978016070 con 0x564f8d6a7130 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f596e7fc640 1 -- 172.21.15.99:0/2802790484 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5978011920 con 0x564f8d6a7130 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.319+0000 7f596e7fc640 1 -- 172.21.15.99:0/2802790484 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59780107b0 con 0x564f8d6a7130 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 -- 172.21.15.99:0/2802790484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 msgr2=0x564f8d6b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f59700b2a10 0x7f59700b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f598000c610 tx=0x7f598000e040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b59a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5978009c50 tx=0x7f597800e6c0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 -- 172.21.15.99:0/2802790484 shutdown_connections 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b59a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 --2- 172.21.15.99:0/2802790484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 -- 172.21.15.99:0/2802790484 >> 172.21.15.99:0/2802790484 conn(0x564f8d6a2e00 msgr2=0x564f8d6a31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 -- 172.21.15.99:0/2802790484 shutdown_connections 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 -- 172.21.15.99:0/2802790484 wait complete. 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.320+0000 7f5991373740 1 -- start start 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 Processor -- start 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564f8d2ef790 con 0x564f8d6a7130 2024-05-14T17:12:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564f8d5c5890 con 0x564f8d6b5ee0 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f8d6b8bb0 con 0x564f8d6a6810 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5991373740 10 client.0 _ll_register_callbacks cb 0x564f8d5c8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45600/0 (socket says 172.21.15.99:45600) 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 -- 172.21.15.99:0/1375955604 learned_addr learned my addr 172.21.15.99:0/1375955604 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f5987fff640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f598cb1c640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 -- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 msgr2=0x564f8d6b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 -- 172.21.15.99:0/1375955604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 msgr2=0x564f8d6b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f59877fe640 1 -- 172.21.15.99:0/1375955604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8d3ea620 con 0x564f8d6a7130 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.321+0000 7f598cb1c640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59877fe640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b2ad0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f597c009900 tx=0x7f597c03b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3775103ef1b307 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59677fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59677fe640 1 -- 172.21.15.99:0/1375955604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c048030 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59677fe640 1 -- 172.21.15.99:0/1375955604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f597c0c5fb0 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5978011920 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59677fe640 1 -- 172.21.15.99:0/1375955604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f597c10b070 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 10 client.16314 fetch_fsmap learned FSMap version 34 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f8d6c4590 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59677fe640 1 -- 172.21.15.99:0/1375955604 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5978011920 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f59677fe640 1 -- 172.21.15.99:0/1375955604 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f597c04b060 con 0x564f8d6a7130 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 10 client.16314 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 20 client.16314 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d6c5a40 0x564f8d6c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 4 client.16314 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f8d61bb80 con 0x564f8d6c5a40 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.322+0000 7f5987fff640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d6c5a40 0x564f8d6c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.323+0000 7f59677fe640 10 client.16314 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.323+0000 7f5987fff640 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d6c5a40 0x564f8d6c7e30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ed0d8ffa7744cd server_cookie=c6bc8d93eef681e in_seq=0 out_seq=0 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.323+0000 7f59677fe640 1 -- 172.21.15.99:0/1375955604 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f8d61bb80 con 0x564f8d6c5a40 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f59677fe640 10 client.16314 handle_command_reply: tid=0 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 client.16314 shutdown 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 msgr2=0x564f8d6b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b2ad0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f597c009900 tx=0x7f597c03b840 comp rx=0 tx=0).stop 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 shutdown_connections 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d6c5a40 0x564f8d6c7e30 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6b5ee0 0x564f8d6b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6a7130 0x564f8d6b2ad0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f59677fe640 0 client.16314 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 --2- 172.21.15.99:0/1375955604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 >> 172.21.15.99:0/1375955604 conn(0x564f8d6a2e00 msgr2=0x564f8d6a8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 shutdown_connections 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 1 -- 172.21.15.99:0/1375955604 wait complete. 2024-05-14T17:12:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.324+0000 7f5991373740 20 client.16314 trim_cache size 0 max 16384 2024-05-14T17:12:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 Processor -- start 2024-05-14T17:12:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 -- start start 2024-05-14T17:12:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a6810 0x564f8d6c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a7130 0x564f8d6cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d6cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aeb60 con 0x564f8d6b5ee0 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59881ac3d0 con 0x564f8d6a6810 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5991373740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aa240 con 0x564f8d6a7130 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f598cb1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d6cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f598cb1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d6cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45614/0 (socket says 172.21.15.99:45614) 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f598cb1c640 1 -- 172.21.15.99:0/2571062555 learned_addr learned my addr 172.21.15.99:0/2571062555 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.339+0000 7f5987fff640 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a6810 0x564f8d6c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f59877fe640 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a7130 0x564f8d6cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f598cb1c640 1 -- 172.21.15.99:0/2571062555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a7130 msgr2=0x564f8d6cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f598cb1c640 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a7130 0x564f8d6cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f598cb1c640 1 -- 172.21.15.99:0/2571062555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a6810 msgr2=0x564f8d6c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f598cb1c640 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a6810 0x564f8d6c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f598cb1c640 1 -- 172.21.15.99:0/2571062555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8d6a2c30 con 0x564f8d6b5ee0 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f59877fe640 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a7130 0x564f8d6cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f598cb1c640 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d6cdc80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5980007680 tx=0x7f5980014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b40840a24395135 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f596dffb640 1 -- 172.21.15.99:0/2571062555 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59800185b0 con 0x564f8d6b5ee0 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f596dffb640 1 -- 172.21.15.99:0/2571062555 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5980028070 con 0x564f8d6b5ee0 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.340+0000 7f596dffb640 1 -- 172.21.15.99:0/2571062555 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980023b70 con 0x564f8d6b5ee0 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 -- 172.21.15.99:0/2571062555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 msgr2=0x564f8d6cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d6cdc80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5980007680 tx=0x7f5980014000 comp rx=0 tx=0).stop 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 -- 172.21.15.99:0/2571062555 shutdown_connections 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d6cdc80 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a7130 0x564f8d6cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 --2- 172.21.15.99:0/2571062555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a6810 0x564f8d6c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 -- 172.21.15.99:0/2571062555 >> 172.21.15.99:0/2571062555 conn(0x564f8d6a2e00 msgr2=0x564f8d6b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 -- 172.21.15.99:0/2571062555 shutdown_connections 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 -- 172.21.15.99:0/2571062555 wait complete. 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 -- start start 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.341+0000 7f5991373740 1 Processor -- start 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a7130 0x564f8d6ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d72f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aa890 con 0x564f8d6b5ee0 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59881abd00 con 0x564f8d6a7130 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59881aaf60 con 0x564f8d6a6810 2024-05-14T17:12:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 10 client.0 _ll_register_callbacks cb 0x7f597c0c5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d72f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d72f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45620/0 (socket says 172.21.15.99:45620) 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 -- 172.21.15.99:0/3824204038 learned_addr learned my addr 172.21.15.99:0/3824204038 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5987fff640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 -- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 msgr2=0x564f8d6b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 -- 172.21.15.99:0/3824204038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a7130 msgr2=0x564f8d6ba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a7130 0x564f8d6ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 -- 172.21.15.99:0/3824204038 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8d72f830 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5987fff640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f598cb1c640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d72f2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5980004a80 tx=0x7f598000c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da24f2b4f6e43792 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f59677fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f59677fe640 1 -- 172.21.15.99:0/3824204038 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598000c220 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f59677fe640 1 -- 172.21.15.99:0/3824204038 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8477+0+0 (secure 0 0 0) 0x7f59800185b0 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5980028070 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f59677fe640 1 -- 172.21.15.99:0/3824204038 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f59800187b0 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 10 client.16320 fetch_fsmap learned FSMap version 34 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f8d7309e0 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.342+0000 7f59677fe640 1 -- 172.21.15.99:0/3824204038 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5980028070 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f59677fe640 1 -- 172.21.15.99:0/3824204038 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f598003e930 con 0x564f8d6b5ee0 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5991373740 10 client.16320 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:12:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5991373740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5991373740 20 client.16320 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5991373740 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d731050 0x564f8d733490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5991373740 4 client.16320 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f8d615360 con 0x564f8d731050 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5987fff640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d731050 0x564f8d733490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f5987fff640 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d731050 0x564f8d733490 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3be87471478d09 server_cookie=9f2c8bdace73ed1f in_seq=0 out_seq=0 2024-05-14T17:12:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.343+0000 7f59677fe640 10 client.16320 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f59677fe640 1 -- 172.21.15.99:0/3824204038 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564f8d615360 con 0x564f8d731050 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f59677fe640 10 client.16320 handle_command_reply: tid=0 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f5991373740 1 client.16320 shutdown 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 msgr2=0x564f8d72f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f5991373740 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d72f2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5980004a80 tx=0x7f598000c460 comp rx=0 tx=0).stop 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 shutdown_connections 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f5991373740 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564f8d731050 0x564f8d733490 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.365+0000 7f5991373740 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f8d6b5ee0 0x564f8d72f2f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f5991373740 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f8d6a7130 0x564f8d6ba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f5991373740 1 --2- 172.21.15.99:0/3824204038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8d6a6810 0x564f8d6b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f59677fe640 0 client.16320 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 >> 172.21.15.99:0/3824204038 conn(0x564f8d6a2e00 msgr2=0x564f8d6ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 shutdown_connections 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f5991373740 1 -- 172.21.15.99:0/3824204038 wait complete. 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f5991373740 20 client.16320 trim_cache size 0 max 16384 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:12:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:12:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f59700b2a10 msgr2=0x7f59700b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f59700b2a10 0x7f59700b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f598000c610 tx=0x7f598000e040 comp rx=0 tx=0).stop 2024-05-14T17:12:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 msgr2=0x7f5988199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 0x7f5988199820 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f597c004770 tx=0x7f597c0042f0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 shutdown_connections 2024-05-14T17:12:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f59700b2a10 0x7f59700b4ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5988107940 0x7f598819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5988102340 0x7f5988199820 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 --2- 172.21.15.99:0/3944575466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5988101980 0x7f59881992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 >> 172.21.15.99:0/3944575466 conn(0x7f5988078ad0 msgr2=0x7f598806a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 shutdown_connections 2024-05-14T17:12:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:10.366+0000 7f598e57f640 1 -- 172.21.15.99:0/3944575466 wait complete. 2024-05-14T17:12:10.471 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 118b3367 --release --await' 2024-05-14T17:12:10.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 118b3367 --release --await 2024-05-14T17:12:10.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 4.91 MB/sec execute 262 sec latency 29369.487 ms 2024-05-14T17:12:10.721 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:12:11.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:10 smithi099 ceph-mon[31851]: pgmap v265: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 378 KiB/s wr, 46 op/s 2024-05-14T17:12:11.015 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:10 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3851083930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:12:11.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:10 smithi193 ceph-mon[35962]: pgmap v265: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 378 KiB/s wr, 46 op/s 2024-05-14T17:12:11.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:10 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3851083930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:12:11.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:10 smithi150 ceph-mon[36204]: pgmap v265: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 378 KiB/s wr, 46 op/s 2024-05-14T17:12:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:10 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3851083930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:12:11.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377395 4.89 MB/sec execute 263 sec latency 30369.518 ms 2024-05-14T17:12:12.152 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:11 smithi099 ceph-mon[31851]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T17:12:12.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:11 smithi150 ceph-mon[36204]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T17:12:12.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:11 smithi193 ceph-mon[35962]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T17:12:12.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 -- 172.21.15.99:0/1711137167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 msgr2=0x7f3b1c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 --2- 172.21.15.99:0/1711137167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c110470 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400b0d0 tx=0x7f3b1400b3e0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 -- 172.21.15.99:0/1711137167 shutdown_connections 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 --2- 172.21.15.99:0/1711137167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c110470 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 --2- 172.21.15.99:0/1711137167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b1c1037f0 0x7f3b1c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 --2- 172.21.15.99:0/1711137167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c1041a0 0x7f3b1c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 -- 172.21.15.99:0/1711137167 >> 172.21.15.99:0/1711137167 conn(0x7f3b1c07a6a0 msgr2=0x7f3b1c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 -- 172.21.15.99:0/1711137167 shutdown_connections 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 -- 172.21.15.99:0/1711137167 wait complete. 2024-05-14T17:12:12.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.420+0000 7f3b2136b700 1 Processor -- start 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 -- start start 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b1c1037f0 0x7f3b1c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c1041a0 0x7f3b1c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1136c0 con 0x7f3b1c1088f0 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c113540 con 0x7f3b1c1041a0 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b2136b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c113840 con 0x7f3b1c1037f0 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45632/0 (socket says 172.21.15.99:45632) 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/2360669142 learned_addr learned my addr 172.21.15.99:0/2360669142 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c1041a0 0x7f3b1c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1affd700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b1c1037f0 0x7f3b1c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/2360669142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b1c1037f0 msgr2=0x7f3b1c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b1c1037f0 0x7f3b1c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/2360669142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c1041a0 msgr2=0x7f3b1c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c1041a0 0x7f3b1c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.421+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/2360669142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c19a040 con 0x7f3b1c1088f0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.422+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c1998e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3b140095c0 tx=0x7f3b14007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.422+0000 7f3b03fff700 1 -- 172.21.15.99:0/2360669142 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1401f070 con 0x7f3b1c1088f0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.422+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c19a2d0 con 0x7f3b1c1088f0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.422+0000 7f3b03fff700 1 -- 172.21.15.99:0/2360669142 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b14003c00 con 0x7f3b1c1088f0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.422+0000 7f3b03fff700 1 -- 172.21.15.99:0/2360669142 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1401b6a0 con 0x7f3b1c1088f0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.422+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c1a65f0 con 0x7f3b1c1088f0 2024-05-14T17:12:12.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.424+0000 7f3b03fff700 1 -- 172.21.15.99:0/2360669142 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3b14022070 con 0x7f3b1c1088f0 2024-05-14T17:12:12.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.425+0000 7f3b03fff700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3b040b29f0 0x7f3b040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.425+0000 7f3b1affd700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3b040b29f0 0x7f3b040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.425+0000 7f3b03fff700 1 -- 172.21.15.99:0/2360669142 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f3b1400f020 con 0x7f3b1c1088f0 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.425+0000 7f3b1affd700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3b040b29f0 0x7f3b040b4eb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c005ea0 tx=0x7f3b0c005e30 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.425+0000 7f3b2bc01b80 1 Processor -- start 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 -- start start 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454ada0 0x56226454b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x5622645454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562264545ad0 0x562264545f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5622644a3d90 con 0x56226454b6c0 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562264493930 con 0x562264545ad0 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562264497970 con 0x56226454ada0 2024-05-14T17:12:12.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x5622645454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x5622645454b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45648/0 (socket says 172.21.15.99:45648) 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/3692337533 learned_addr learned my addr 172.21.15.99:0/3692337533 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1affd700 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454ada0 0x56226454b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562264545ad0 0x562264545f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/3692337533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454ada0 msgr2=0x56226454b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454ada0 0x56226454b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/3692337533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562264545ad0 msgr2=0x562264545f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562264545ad0 0x562264545f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/3692337533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622643e39d0 con 0x56226454b6c0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562264545ad0 0x562264545f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.426+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x5622645454b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000bd30 tx=0x7f3b100106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969607bd4d569b49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b017fa700 1 -- 172.21.15.99:0/3692337533 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10010e10 con 0x56226454b6c0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b017fa700 1 -- 172.21.15.99:0/3692337533 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b100044e0 con 0x56226454b6c0 2024-05-14T17:12:12.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b017fa700 1 -- 172.21.15.99:0/3692337533 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10011440 con 0x56226454b6c0 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3692337533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 msgr2=0x5622645454b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x5622645454b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000bd30 tx=0x7f3b100106b0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3692337533 shutdown_connections 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562264545ad0 0x562264545f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454ada0 0x56226454b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3692337533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x5622645454b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3692337533 >> 172.21.15.99:0/3692337533 conn(0x562264470350 msgr2=0x56226453fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3692337533 shutdown_connections 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.427+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3692337533 wait complete. 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 -- start start 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 Processor -- start 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x562264551420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x56226454e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x56226454ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56226439a0a0 con 0x56226454b6c0 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5622644358c0 con 0x56226454ada0 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562264552100 con 0x562264545ad0 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b2bc01b80 10 client.0 _ll_register_callbacks cb 0x56226442a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x56226454ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b1b7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x56226454ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45656/0 (socket says 172.21.15.99:45656) 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/3740158887 learned_addr learned my addr 172.21.15.99:0/3740158887 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x56226454e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.428+0000 7f3b1affd700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x562264551420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 msgr2=0x562264551420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x562264551420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/3740158887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 msgr2=0x56226454e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x56226454e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b1b7fe700 1 -- 172.21.15.99:0/3740158887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562264432790 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x56226454ed00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009fd0 tx=0x7f3b140aaac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c38abe1c880911 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3afeff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3afeff5700 1 -- 172.21.15.99:0/3740158887 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b140a6390 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3afeff5700 1 -- 172.21.15.99:0/3740158887 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f3b140b3540 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b100044e0 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3afeff5700 1 -- 172.21.15.99:0/3740158887 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3b140b3310 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b2bc01b80 10 client.16332 fetch_fsmap learned FSMap version 34 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56226455c7d0 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3afeff5700 1 -- 172.21.15.99:0/3740158887 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b100044e0 con 0x56226454b6c0 2024-05-14T17:12:12.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.429+0000 7f3afeff5700 1 -- 172.21.15.99:0/3740158887 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f3b14011070 con 0x56226454b6c0 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b2bc01b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b2bc01b80 10 client.16332 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b2bc01b80 20 client.16332 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56226455ede0 0x5622645611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b2bc01b80 4 client.16332 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562264511460 con 0x56226455ede0 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b1affd700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56226455ede0 0x5622645611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3afeff5700 10 client.16332 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.430+0000 7f3b1affd700 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56226455ede0 0x5622645611d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc9d3006352ad0a server_cookie=7f0f5f201820eb79 in_seq=0 out_seq=0 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3afeff5700 1 -- 172.21.15.99:0/3740158887 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562264511460 con 0x56226455ede0 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3afeff5700 10 client.16332 handle_command_reply: tid=0 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 client.16332 shutdown 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 msgr2=0x56226454ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x56226454ed00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009fd0 tx=0x7f3b140aaac0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 shutdown_connections 2024-05-14T17:12:12.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56226455ede0 0x5622645611d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454b6c0 0x56226454ed00 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x562264551420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/3740158887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x56226454e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3afeff5700 0 client.16332 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 >> 172.21.15.99:0/3740158887 conn(0x562264470350 msgr2=0x56226453a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.431+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 shutdown_connections 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.432+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/3740158887 wait complete. 2024-05-14T17:12:12.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.432+0000 7f3b2bc01b80 20 client.16332 trim_cache size 0 max 16384 2024-05-14T17:12:12.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 Processor -- start 2024-05-14T17:12:12.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 -- start start 2024-05-14T17:12:12.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x56226455b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 0x56226453a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454b6c0 0x56226454c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1ae320 con 0x56226454ada0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1abb50 con 0x56226454b6c0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1a99c0 con 0x562264545ad0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 0x56226453a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 0x56226453a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45660/0 (socket says 172.21.15.99:45660) 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/1203865014 learned_addr learned my addr 172.21.15.99:0/1203865014 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454b6c0 0x56226454c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1affd700 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x56226455b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/1203865014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 msgr2=0x56226455b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x56226455b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/1203865014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454b6c0 msgr2=0x56226454c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454b6c0 0x56226454c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.451+0000 7f3b1a7fc700 1 -- 172.21.15.99:0/1203865014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226454e590 con 0x56226454ada0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 0x56226453a8e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b6c0 tx=0x7f3b10005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2e5c21888e228b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b00ff9700 1 -- 172.21.15.99:0/1203865014 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1002a070 con 0x56226454ada0 2024-05-14T17:12:12.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b00ff9700 1 -- 172.21.15.99:0/1203865014 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b100054f0 con 0x56226454ada0 2024-05-14T17:12:12.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b00ff9700 1 -- 172.21.15.99:0/1203865014 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10011440 con 0x56226454ada0 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/1203865014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 msgr2=0x56226453a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 0x56226453a8e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b6c0 tx=0x7f3b10005d00 comp rx=0 tx=0).stop 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/1203865014 shutdown_connections 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454b6c0 0x56226454c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264545ad0 0x56226455b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/1203865014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56226454ada0 0x56226453a8e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/1203865014 >> 172.21.15.99:0/1203865014 conn(0x562264470350 msgr2=0x56226454a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.452+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/1203865014 shutdown_connections 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/1203865014 wait complete. 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 -- start start 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 Processor -- start 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 0x562264554360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x5622645548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454b6c0 0x562264554de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1aa010 con 0x562264545ad0 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1ab480 con 0x56226454ada0 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1aa6e0 con 0x56226454b6c0 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b2bc01b80 10 client.0 _ll_register_callbacks cb 0x56226442a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 0x562264554360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 0x562264554360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45664/0 (socket says 172.21.15.99:45664) 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 -- 172.21.15.99:0/486931117 learned_addr learned my addr 172.21.15.99:0/486931117 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:12.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x5622645548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454b6c0 0x562264554de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 -- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454b6c0 msgr2=0x562264554de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454b6c0 0x562264554de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 -- 172.21.15.99:0/486931117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 msgr2=0x5622645548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x5622645548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.453+0000 7f3b1affd700 1 -- 172.21.15.99:0/486931117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622645c9800 con 0x562264545ad0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b1b7fe700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454b6c0 0x562264554de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x5622645548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b1affd700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 0x562264554360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00afd0 tx=0x7f3b0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0c5a3ee3c426dc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3afeff5700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3afeff5700 1 -- 172.21.15.99:0/486931117 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c010a10 con 0x562264545ad0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b100054f0 con 0x562264545ad0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3afeff5700 1 -- 172.21.15.99:0/486931117 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8565+0+0 (secure 0 0 0) 0x7f3b0c010bb0 con 0x562264545ad0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3afeff5700 1 -- 172.21.15.99:0/486931117 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3b0c010db0 con 0x562264545ad0 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b2bc01b80 10 client.16338 fetch_fsmap learned FSMap version 34 2024-05-14T17:12:12.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5622645cab60 con 0x562264545ad0 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3afeff5700 1 -- 172.21.15.99:0/486931117 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b100054f0 con 0x562264545ad0 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3afeff5700 1 -- 172.21.15.99:0/486931117 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f3b0c037420 con 0x562264545ad0 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b2bc01b80 10 client.16338 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b2bc01b80 10 client.16338 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.454+0000 7f3b2bc01b80 20 client.16338 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.455+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5622645cb270 0x5622645cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.455+0000 7f3b2bc01b80 4 client.16338 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "118b3367", "release": true, "await": true}] 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.455+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "118b3367", "release": true, "await": true}) -- 0x5622644a5ce0 con 0x5622645cb270 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.455+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5622645cb270 0x5622645cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.455+0000 7f3b1a7fc700 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5622645cb270 0x5622645cd690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5553635057cc04 server_cookie=89fa038fd219cf0d in_seq=0 out_seq=0 2024-05-14T17:12:12.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.455+0000 7f3afeff5700 10 client.16338 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3afeff5700 1 -- 172.21.15.99:0/486931117 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5622644a5ce0 con 0x5622645cb270 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3afeff5700 10 client.16338 handle_command_reply: tid=0 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 client.16338 shutdown 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 msgr2=0x562264554360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 0x562264554360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00afd0 tx=0x7f3b0c0047c0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 shutdown_connections 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56226454b6c0 0x562264554de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5622645cb270 0x5622645cd690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562264545ad0 0x562264554360 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 --2- 172.21.15.99:0/486931117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56226454ada0 0x5622645548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 >> 172.21.15.99:0/486931117 conn(0x562264470350 msgr2=0x562264549980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3afeff5700 0 client.16338 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 shutdown_connections 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 1 -- 172.21.15.99:0/486931117 wait complete. 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.460+0000 7f3b2bc01b80 20 client.16338 trim_cache size 0 max 16384 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 34, 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:12:12.461 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 28, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "118b3367": { 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 28, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 151.3, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:12:12.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:12:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3b040b29f0 msgr2=0x7f3b040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3b040b29f0 0x7f3b040b4eb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c005ea0 tx=0x7f3b0c005e30 comp rx=0 tx=0).stop 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 msgr2=0x7f3b1c1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c1998e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3b140095c0 tx=0x7f3b14007a20 comp rx=0 tx=0).stop 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 shutdown_connections 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3b040b29f0 0x7f3b040b4eb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b1c1088f0 0x7f3b1c1998e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b1c1037f0 0x7f3b1c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 --2- 172.21.15.99:0/2360669142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b1c1041a0 0x7f3b1c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 >> 172.21.15.99:0/2360669142 conn(0x7f3b1c07a6a0 msgr2=0x7f3b1c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 shutdown_connections 2024-05-14T17:12:12.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:12:12.461+0000 7f3b2136b700 1 -- 172.21.15.99:0/2360669142 wait complete. 2024-05-14T17:12:12.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377480 4.87 MB/sec execute 264 sec latency 31292.434 ms 2024-05-14T17:12:13.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:12 smithi099 ceph-mon[31851]: pgmap v267: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 427 KiB/s wr, 46 op/s 2024-05-14T17:12:13.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '118b3367', total seconds elapsed: 36.8 2024-05-14T17:12:13.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:12 smithi150 ceph-mon[36204]: pgmap v267: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 427 KiB/s wr, 46 op/s 2024-05-14T17:12:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:12 smithi193 ceph-mon[35962]: pgmap v267: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 427 KiB/s wr, 46 op/s 2024-05-14T17:12:13.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377666 4.85 MB/sec execute 265 sec latency 872.645 ms 2024-05-14T17:12:14.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 377666 4.83 MB/sec execute 266 sec latency 1872.748 ms 2024-05-14T17:12:15.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:14 smithi150 ceph-mon[36204]: pgmap v268: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 801 KiB/s wr, 51 op/s 2024-05-14T17:12:15.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:14 smithi150 ceph-mon[36204]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T17:12:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:14 smithi099 ceph-mon[31851]: pgmap v268: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 801 KiB/s wr, 51 op/s 2024-05-14T17:12:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:14 smithi099 ceph-mon[31851]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T17:12:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:14 smithi193 ceph-mon[35962]: pgmap v268: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 801 KiB/s wr, 51 op/s 2024-05-14T17:12:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:14 smithi193 ceph-mon[35962]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T17:12:15.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 379885 4.87 MB/sec execute 267 sec latency 1928.638 ms 2024-05-14T17:12:16.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.87 MB/sec execute 268 sec latency 180.154 ms 2024-05-14T17:12:17.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:16 smithi150 ceph-mon[36204]: pgmap v270: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1001 KiB/s wr, 64 op/s 2024-05-14T17:12:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:16 smithi099 ceph-mon[31851]: pgmap v270: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1001 KiB/s wr, 64 op/s 2024-05-14T17:12:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:16 smithi193 ceph-mon[35962]: pgmap v270: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1001 KiB/s wr, 64 op/s 2024-05-14T17:12:17.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.85 MB/sec execute 269 sec latency 1180.298 ms 2024-05-14T17:12:18.541 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.83 MB/sec execute 270 sec latency 2180.410 ms 2024-05-14T17:12:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:18 smithi150 ceph-mon[36204]: pgmap v271: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 624 KiB/s wr, 44 op/s 2024-05-14T17:12:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:18 smithi099 ceph-mon[31851]: pgmap v271: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 624 KiB/s wr, 44 op/s 2024-05-14T17:12:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:18 smithi193 ceph-mon[35962]: pgmap v271: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 624 KiB/s wr, 44 op/s 2024-05-14T17:12:19.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 381255 4.82 MB/sec execute 271 sec latency 3180.514 ms 2024-05-14T17:12:20.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 383492 4.85 MB/sec execute 272 sec latency 3235.723 ms 2024-05-14T17:12:21.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:20 smithi150 ceph-mon[36204]: pgmap v272: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-14T17:12:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:20 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:20 smithi099 ceph-mon[31851]: pgmap v272: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-14T17:12:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:20 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:20 smithi193 ceph-mon[35962]: pgmap v272: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 97 op/s 2024-05-14T17:12:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:20 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:21.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.85 MB/sec execute 273 sec latency 86.407 ms 2024-05-14T17:12:22.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.83 MB/sec execute 274 sec latency 1086.518 ms 2024-05-14T17:12:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:22 smithi150 ceph-mon[36204]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-05-14T17:12:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:22 smithi099 ceph-mon[31851]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-05-14T17:12:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:22 smithi193 ceph-mon[35962]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 -- 172.21.15.99:0/2154132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800107a50 msgr2=0x7fa800107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 --2- 172.21.15.99:0/2154132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800107a50 0x7fa800107e30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009930 tx=0x7fa7f402a420 comp rx=0 tx=0).stop 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 -- 172.21.15.99:0/2154132812 shutdown_connections 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 --2- 172.21.15.99:0/2154132812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa800101930 0x7fa80010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 --2- 172.21.15.99:0/2154132812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa800100f70 0x7fa8001013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 --2- 172.21.15.99:0/2154132812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800107a50 0x7fa800107e30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 -- 172.21.15.99:0/2154132812 >> 172.21.15.99:0/2154132812 conn(0x7fa800077ee0 msgr2=0x7fa8000ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 -- 172.21.15.99:0/2154132812 shutdown_connections 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.465+0000 7fa806ea9640 1 -- 172.21.15.99:0/2154132812 wait complete. 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 Processor -- start 2024-05-14T17:12:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 -- start start 2024-05-14T17:12:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 0x7fa800102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa800101930 0x7fa800102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa800107a50 0x7fa800102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa800112dd0 con 0x7fa800100f70 2024-05-14T17:12:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa800112c50 con 0x7fa800107a50 2024-05-14T17:12:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa806ea9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa800112f50 con 0x7fa800101930 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 0x7fa800102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 0x7fa800102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42710/0 (socket says 172.21.15.99:42710) 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 -- 172.21.15.99:0/275034901 learned_addr learned my addr 172.21.15.99:0/275034901 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa805446640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa800107a50 0x7fa800102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa7fffff640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa800101930 0x7fa800102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 -- 172.21.15.99:0/275034901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa800101930 msgr2=0x7fa800102690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa800101930 0x7fa800102690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 -- 172.21.15.99:0/275034901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa800107a50 msgr2=0x7fa800102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa800107a50 0x7fa800102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 -- 172.21.15.99:0/275034901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8001a7670 con 0x7fa800100f70 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.466+0000 7fa804c45640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 0x7fa800102150 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005940 tx=0x7fa7f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.467+0000 7fa7fdffb640 1 -- 172.21.15.99:0/275034901 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f403f030 con 0x7fa800100f70 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.467+0000 7fa7fdffb640 1 -- 172.21.15.99:0/275034901 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f4003aa0 con 0x7fa800100f70 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.467+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8001a7900 con 0x7fa800100f70 2024-05-14T17:12:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.467+0000 7fa7fdffb640 1 -- 172.21.15.99:0/275034901 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4035070 con 0x7fa800100f70 2024-05-14T17:12:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.467+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001a7e40 con 0x7fa800100f70 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.469+0000 7fa7fdffb640 1 -- 172.21.15.99:0/275034901 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa7f4042030 con 0x7fa800100f70 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.469+0000 7fa809bc5740 1 Processor -- start 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 -- start start 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 0x55e557921cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e557922210 0x55e55792c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e55792eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e557894c30 con 0x55e5579218f0 2024-05-14T17:12:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e5578847d0 con 0x55e55792cad0 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa809bc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e557888810 con 0x55e557922210 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 0x55e557921cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 0x55e557921cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42726/0 (socket says 172.21.15.99:42726) 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 -- 172.21.15.99:0/3095090574 learned_addr learned my addr 172.21.15.99:0/3095090574 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa804c45640 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e557922210 0x55e55792c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa7fffff640 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e55792eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 -- 172.21.15.99:0/3095090574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e557922210 msgr2=0x55e55792c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e557922210 0x55e55792c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 -- 172.21.15.99:0/3095090574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 msgr2=0x55e55792eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e55792eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa7fdffb640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7f00b2a90 0x7fa7f00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 -- 172.21.15.99:0/3095090574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e557599210 con 0x55e5579218f0 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa7fffff640 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e55792eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa805446640 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 0x55e557921cd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8007ec0 tx=0x7fa7f800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd2eca862157429 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.470+0000 7fa7fdffb640 1 -- 172.21.15.99:0/275034901 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa7f4102d50 con 0x7fa800100f70 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa805446640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7f00b2a90 0x7fa7f00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa7eaffd640 1 -- 172.21.15.99:0/3095090574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80108e0 con 0x55e5579218f0 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa7eaffd640 1 -- 172.21.15.99:0/3095090574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f8010de0 con 0x55e5579218f0 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa7eaffd640 1 -- 172.21.15.99:0/3095090574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8017830 con 0x55e5579218f0 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 -- 172.21.15.99:0/3095090574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 msgr2=0x55e557921cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa805446640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7f00b2a90 0x7fa7f00b4f50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7f801a040 tx=0x7fa7f8017bd0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 0x55e557921cd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8007ec0 tx=0x7fa7f800f790 comp rx=0 tx=0).stop 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 -- 172.21.15.99:0/3095090574 shutdown_connections 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e55792eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e557922210 0x55e55792c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 --2- 172.21.15.99:0/3095090574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e5579218f0 0x55e557921cd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.471+0000 7fa809bc5740 1 -- 172.21.15.99:0/3095090574 >> 172.21.15.99:0/3095090574 conn(0x55e55791de00 msgr2=0x55e55791e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 -- 172.21.15.99:0/3095090574 shutdown_connections 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 -- 172.21.15.99:0/3095090574 wait complete. 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 -- start start 2024-05-14T17:12:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 Processor -- start 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55792c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e5579296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e557929c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e55756a790 con 0x55e557922210 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e557840890 con 0x55e55792cad0 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e55792f7a0 con 0x55e5579218f0 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa809bc5740 10 client.0 _ll_register_callbacks cb 0x55e557843d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e5579296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.472+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e5579296d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42730/0 (socket says 172.21.15.99:42730) 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 -- 172.21.15.99:0/2412191868 learned_addr learned my addr 172.21.15.99:0/2412191868 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa804c45640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55792c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa805446640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e557929c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 -- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 msgr2=0x55e55792c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55792c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 -- 172.21.15.99:0/2412191868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 msgr2=0x55e557929c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e557929c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 -- 172.21.15.99:0/2412191868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e557665620 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7fffff640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e5579296d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00e6e0 tx=0x7fa7ec00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e9e9f8685b26d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7dffff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7dffff640 1 -- 172.21.15.99:0/2412191868 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec01c060 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7dffff640 1 -- 172.21.15.99:0/2412191868 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa7ec009a80 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f8010de0 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa7dffff640 1 -- 172.21.15.99:0/2412191868 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec009c80 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa809bc5740 10 client.16350 fetch_fsmap learned FSMap version 35 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.473+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e55793b140 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa7dffff640 1 -- 172.21.15.99:0/2412191868 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f8010de0 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa7dffff640 1 -- 172.21.15.99:0/2412191868 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7fa7ec004ad0 con 0x55e557922210 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa809bc5740 10 client.16350 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T17:12:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa809bc5740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa809bc5740 20 client.16350 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e55793c6a0 0x55e55793ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa809bc5740 4 client.16350 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e557896b80 con 0x55e55793c6a0 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa804c45640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e55793c6a0 0x55e55793ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa7dffff640 10 client.16350 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.474+0000 7fa804c45640 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e55793c6a0 0x55e55793ead0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8424fd1aa0380c40 server_cookie=5aa4c0cc824a893 in_seq=0 out_seq=0 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa7dffff640 1 -- 172.21.15.99:0/2412191868 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e557896b80 con 0x55e55793c6a0 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa7dffff640 10 client.16350 handle_command_reply: tid=0 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 client.16350 shutdown 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 msgr2=0x55e5579296d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e5579296d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00e6e0 tx=0x7fa7ec00eba0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 shutdown_connections 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e55793c6a0 0x55e55793ead0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e557929c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e5579296d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa7dffff640 0 client.16350 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2412191868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55792c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 >> 172.21.15.99:0/2412191868 conn(0x55e55791de00 msgr2=0x55e55791d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.475+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 shutdown_connections 2024-05-14T17:12:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.476+0000 7fa809bc5740 1 -- 172.21.15.99:0/2412191868 wait complete. 2024-05-14T17:12:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.476+0000 7fa809bc5740 20 client.16350 trim_cache size 0 max 16384 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 Processor -- start 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 -- start start 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55791d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557938670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e5579443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b3080 con 0x55e557922210 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b0920 con 0x55e55792cad0 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa809bc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001ae790 con 0x55e5579218f0 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557938670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557938670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42736/0 (socket says 172.21.15.99:42736) 2024-05-14T17:12:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 -- 172.21.15.99:0/2228883055 learned_addr learned my addr 172.21.15.99:0/2228883055 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa804c45640 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55791d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 -- 172.21.15.99:0/2228883055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 msgr2=0x55e55791d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55791d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 -- 172.21.15.99:0/2228883055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 msgr2=0x55e5579443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e5579443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 -- 172.21.15.99:0/2228883055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e557928680 con 0x55e557922210 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.491+0000 7fa7fffff640 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557938670 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00c380 tx=0x7fa7ec00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31477178cdb86dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa7ea7fc640 1 -- 172.21.15.99:0/2228883055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00c420 con 0x55e557922210 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa7ea7fc640 1 -- 172.21.15.99:0/2228883055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec009880 con 0x55e557922210 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa7ea7fc640 1 -- 172.21.15.99:0/2228883055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec004660 con 0x55e557922210 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 -- 172.21.15.99:0/2228883055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 msgr2=0x55e557938670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557938670 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00c380 tx=0x7fa7ec00ceb0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 -- 172.21.15.99:0/2228883055 shutdown_connections 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e55792cad0 0x55e5579443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557938670 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 --2- 172.21.15.99:0/2228883055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5579218f0 0x55e55791d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 -- 172.21.15.99:0/2228883055 >> 172.21.15.99:0/2228883055 conn(0x55e55791de00 msgr2=0x55e55791d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 -- 172.21.15.99:0/2228883055 shutdown_connections 2024-05-14T17:12:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.492+0000 7fa809bc5740 1 -- 172.21.15.99:0/2228883055 wait complete. 2024-05-14T17:12:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 -- start start 2024-05-14T17:12:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 Processor -- start 2024-05-14T17:12:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5579218f0 0x55e557932bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557933130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55792cad0 0x55e5579a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001aede0 con 0x55e557922210 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b0250 con 0x55e5579218f0 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001af4b0 con 0x55e55792cad0 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa809bc5740 10 client.0 _ll_register_callbacks cb 0x7fa7ec00cde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557933130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557933130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42742/0 (socket says 172.21.15.99:42742) 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 -- 172.21.15.99:0/1329480735 learned_addr learned my addr 172.21.15.99:0/1329480735 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa804c45640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5579218f0 0x55e557932bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa805446640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55792cad0 0x55e5579a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 -- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55792cad0 msgr2=0x55e5579a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55792cad0 0x55e5579a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 -- 172.21.15.99:0/1329480735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5579218f0 msgr2=0x55e557932bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5579218f0 0x55e557932bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.493+0000 7fa7fffff640 1 -- 172.21.15.99:0/1329480735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5579a64e0 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7fffff640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557933130 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec005bd0 tx=0x7fa7ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6218cfb38d769141 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7dffff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7dffff640 1 -- 172.21.15.99:0/1329480735 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec036020 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7dffff640 1 -- 172.21.15.99:0/1329480735 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa7ec004660 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec009880 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7dffff640 1 -- 172.21.15.99:0/1329480735 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec041050 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 10 client.16356 fetch_fsmap learned FSMap version 35 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e5579a76f0 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7dffff640 1 -- 172.21.15.99:0/1329480735 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec009880 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa7dffff640 1 -- 172.21.15.99:0/1329480735 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7fa7ec017440 con 0x55e557922210 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 10 client.16356 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T17:12:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 20 client.16356 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e5579a7cf0 0x55e5579aa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 4 client.16356 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.494+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e557890360 con 0x55e5579a7cf0 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.495+0000 7fa804c45640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e5579a7cf0 0x55e5579aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.495+0000 7fa804c45640 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e5579a7cf0 0x55e5579aa110 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c324804c163752a0 server_cookie=7a060d2a5e7e5332 in_seq=0 out_seq=0 2024-05-14T17:12:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.495+0000 7fa7dffff640 10 client.16356 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa7dffff640 1 -- 172.21.15.99:0/1329480735 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e557890360 con 0x55e5579a7cf0 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa7dffff640 10 client.16356 handle_command_reply: tid=0 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 client.16356 shutdown 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 msgr2=0x55e557933130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557933130 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec005bd0 tx=0x7fa7ec009bf0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 shutdown_connections 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55e5579a7cf0 0x55e5579aa110 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55792cad0 0x55e5579a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa7dffff640 0 client.16356 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e557922210 0x55e557933130 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 --2- 172.21.15.99:0/1329480735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5579218f0 0x55e557932bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 >> 172.21.15.99:0/1329480735 conn(0x55e55791de00 msgr2=0x55e557937660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 shutdown_connections 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 1 -- 172.21.15.99:0/1329480735 wait complete. 2024-05-14T17:12:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.535+0000 7fa809bc5740 20 client.16356 trim_cache size 0 max 16384 2024-05-14T17:12:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:12:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:12:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7f00b2a90 msgr2=0x7fa7f00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7f00b2a90 0x7fa7f00b4f50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7f801a040 tx=0x7fa7f8017bd0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 msgr2=0x7fa800102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 0x7fa800102150 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005940 tx=0x7fa7f4004270 comp rx=0 tx=0).stop 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 shutdown_connections 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7f00b2a90 0x7fa7f00b4f50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa800107a50 0x7fa800102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa800101930 0x7fa800102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 --2- 172.21.15.99:0/275034901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800100f70 0x7fa800102150 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 >> 172.21.15.99:0/275034901 conn(0x7fa800077ee0 msgr2=0x7fa800105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 shutdown_connections 2024-05-14T17:12:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:23.536+0000 7fa806ea9640 1 -- 172.21.15.99:0/275034901 wait complete. 2024-05-14T17:12:23.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.82 MB/sec execute 275 sec latency 2086.597 ms 2024-05-14T17:12:24.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 384796 4.80 MB/sec execute 276 sec latency 3086.683 ms 2024-05-14T17:12:25.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:24 smithi150 ceph-mon[36204]: pgmap v274: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 995 KiB/s wr, 109 op/s 2024-05-14T17:12:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:24 smithi099 ceph-mon[31851]: pgmap v274: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 995 KiB/s wr, 109 op/s 2024-05-14T17:12:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:24 smithi193 ceph-mon[35962]: pgmap v274: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 995 KiB/s wr, 109 op/s 2024-05-14T17:12:25.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 386930 4.83 MB/sec execute 277 sec latency 3108.859 ms 2024-05-14T17:12:26.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.83 MB/sec execute 278 sec latency 147.224 ms 2024-05-14T17:12:27.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:26 smithi150 ceph-mon[36204]: pgmap v275: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 KiB/s wr, 101 op/s 2024-05-14T17:12:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:26 smithi099 ceph-mon[31851]: pgmap v275: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 KiB/s wr, 101 op/s 2024-05-14T17:12:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:26 smithi193 ceph-mon[35962]: pgmap v275: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 KiB/s wr, 101 op/s 2024-05-14T17:12:27.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.82 MB/sec execute 279 sec latency 1147.300 ms 2024-05-14T17:12:28.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.80 MB/sec execute 280 sec latency 2147.344 ms 2024-05-14T17:12:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:28 smithi150 ceph-mon[36204]: pgmap v276: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 829 KiB/s wr, 91 op/s 2024-05-14T17:12:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:28 smithi099 ceph-mon[31851]: pgmap v276: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 829 KiB/s wr, 91 op/s 2024-05-14T17:12:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:28 smithi193 ceph-mon[35962]: pgmap v276: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 829 KiB/s wr, 91 op/s 2024-05-14T17:12:29.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 388368 4.78 MB/sec execute 281 sec latency 3147.451 ms 2024-05-14T17:12:30.542 INFO:tasks.workunit.client.0.smithi099.stdout: 1 390559 4.81 MB/sec execute 282 sec latency 3203.641 ms 2024-05-14T17:12:31.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:30 smithi150 ceph-mon[36204]: pgmap v277: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-14T17:12:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:30 smithi099 ceph-mon[31851]: pgmap v277: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-14T17:12:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:30 smithi193 ceph-mon[35962]: pgmap v277: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-14T17:12:31.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.81 MB/sec execute 283 sec latency 37.384 ms 2024-05-14T17:12:32.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.80 MB/sec execute 284 sec latency 1037.434 ms 2024-05-14T17:12:33.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:32 smithi150 ceph-mon[36204]: pgmap v278: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 643 KiB/s wr, 87 op/s 2024-05-14T17:12:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:32 smithi099 ceph-mon[31851]: pgmap v278: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 643 KiB/s wr, 87 op/s 2024-05-14T17:12:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:32 smithi193 ceph-mon[35962]: pgmap v278: 97 pgs: 97 active+clean; 99 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 643 KiB/s wr, 87 op/s 2024-05-14T17:12:33.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.78 MB/sec execute 285 sec latency 2037.535 ms 2024-05-14T17:12:34.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 391911 4.76 MB/sec execute 286 sec latency 3037.621 ms 2024-05-14T17:12:35.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:34 smithi150 ceph-mon[36204]: pgmap v279: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 954 KiB/s wr, 126 op/s 2024-05-14T17:12:35.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:34 smithi099 ceph-mon[31851]: pgmap v279: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 954 KiB/s wr, 126 op/s 2024-05-14T17:12:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:34 smithi193 ceph-mon[35962]: pgmap v279: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 954 KiB/s wr, 126 op/s 2024-05-14T17:12:35.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 393581 4.78 MB/sec execute 287 sec latency 3091.661 ms 2024-05-14T17:12:36.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 393581 4.76 MB/sec execute 288 sec latency 1363.606 ms 2024-05-14T17:12:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 -- 172.21.15.99:0/1522573013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc102310 msgr2=0x7faedc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 --2- 172.21.15.99:0/1522573013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc102310 0x7faedc10e820 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faed4009cd0 tx=0x7faed402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 -- 172.21.15.99:0/1522573013 shutdown_connections 2024-05-14T17:12:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 --2- 172.21.15.99:0/1522573013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc102310 0x7faedc10e820 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 --2- 172.21.15.99:0/1522573013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faedc101950 0x7faedc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 --2- 172.21.15.99:0/1522573013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faedc107960 0x7faedc107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 -- 172.21.15.99:0/1522573013 >> 172.21.15.99:0/1522573013 conn(0x7faedc078a60 msgr2=0x7faedc078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 -- 172.21.15.99:0/1522573013 shutdown_connections 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.633+0000 7faee16f8640 1 -- 172.21.15.99:0/1522573013 wait complete. 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 Processor -- start 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 -- start start 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faedc101950 0x7faedc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faedc102310 0x7faedc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 0x7faedc1a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faedc111c20 con 0x7faedc107960 2024-05-14T17:12:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faedc111aa0 con 0x7faedc101950 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.634+0000 7faee16f8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faedc111da0 con 0x7faedc102310 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 0x7faedc1a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 0x7faedc1a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52670/0 (socket says 172.21.15.99:52670) 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 -- 172.21.15.99:0/1116037773 learned_addr learned my addr 172.21.15.99:0/1116037773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faed27fc640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faedc102310 0x7faedc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedaffd640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faedc101950 0x7faedc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 -- 172.21.15.99:0/1116037773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faedc102310 msgr2=0x7faedc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faedc102310 0x7faedc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 -- 172.21.15.99:0/1116037773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faedc101950 msgr2=0x7faedc199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faedc101950 0x7faedc199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 -- 172.21.15.99:0/1116037773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc1a5580 con 0x7faedc107960 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faedb7fe640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 0x7faedc1a5040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faed4005cc0 tx=0x7faed4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faed8ff9640 1 -- 172.21.15.99:0/1116037773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4004450 con 0x7faedc107960 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faed8ff9640 1 -- 172.21.15.99:0/1116037773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed4031c90 con 0x7faedc107960 2024-05-14T17:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc1a5750 con 0x7faedc107960 2024-05-14T17:12:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faed8ff9640 1 -- 172.21.15.99:0/1116037773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed403b400 con 0x7faedc107960 2024-05-14T17:12:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.635+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc1a5c40 con 0x7faedc107960 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faed8ff9640 1 -- 172.21.15.99:0/1116037773 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faed4004680 con 0x7faedc107960 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 Processor -- start 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 -- start start 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e1637cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e1642590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 0x5632e1644ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5632e15aac30 con 0x5632e1642ad0 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5632e159a7d0 con 0x5632e1638210 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.638+0000 7faee4410740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632e159e810 con 0x5632e16378f0 2024-05-14T17:12:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 0x5632e1644ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faedaffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e1642590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faedaffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e1642590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:60730/0 (socket says 172.21.15.99:60730) 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 0x5632e1644ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52676/0 (socket says 172.21.15.99:52676) 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faedaffd640 1 -- 172.21.15.99:0/3206723575 learned_addr learned my addr 172.21.15.99:0/3206723575 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e1637cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed8ff9640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faec80b2a10 0x7faec80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 -- 172.21.15.99:0/3206723575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 msgr2=0x5632e1637cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e1637cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faedb7fe640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faec80b2a10 0x7faec80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 -- 172.21.15.99:0/3206723575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 msgr2=0x5632e1642590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e1642590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 -- 172.21.15.99:0/3206723575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632e12af210 con 0x5632e1642ad0 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed27fc640 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 0x5632e1644ec0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faedc06b780 tx=0x7faec400a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582668642286fb92 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed8ff9640 1 -- 172.21.15.99:0/1116037773 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7faed402e030 con 0x7faedc107960 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed17fa640 1 -- 172.21.15.99:0/3206723575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4004020 con 0x5632e1642ad0 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed17fa640 1 -- 172.21.15.99:0/3206723575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec40054d0 con 0x5632e1642ad0 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.639+0000 7faed17fa640 1 -- 172.21.15.99:0/3206723575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4004c10 con 0x5632e1642ad0 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 -- 172.21.15.99:0/3206723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 msgr2=0x5632e1644ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 0x5632e1644ec0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faedc06b780 tx=0x7faec400a640 comp rx=0 tx=0).stop 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faedb7fe640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faec80b2a10 0x7faec80b4ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faed40c6b10 tx=0x7faed40c6a80 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 -- 172.21.15.99:0/3206723575 shutdown_connections 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1642ad0 0x5632e1644ec0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e1642590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 --2- 172.21.15.99:0/3206723575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e1637cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 -- 172.21.15.99:0/3206723575 >> 172.21.15.99:0/3206723575 conn(0x5632e1633e00 msgr2=0x5632e16341e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 -- 172.21.15.99:0/3206723575 shutdown_connections 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.640+0000 7faee4410740 1 -- 172.21.15.99:0/3206723575 wait complete. 2024-05-14T17:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 -- start start 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 Processor -- start 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e16424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 0x5632e163f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e163fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5632e1280790 con 0x5632e1638210 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5632e1556890 con 0x5632e1642ad0 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632e16457a0 con 0x5632e16378f0 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faee4410740 10 client.0 _ll_register_callbacks cb 0x5632e1559d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 0x5632e163f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 0x5632e163f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52686/0 (socket says 172.21.15.99:52686) 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 -- 172.21.15.99:0/1646315196 learned_addr learned my addr 172.21.15.99:0/1646315196 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faed27fc640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e16424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 -- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 msgr2=0x5632e16424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e16424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 -- 172.21.15.99:0/1646315196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 msgr2=0x5632e163fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faed27fc640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e163fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e163fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faedaffd640 1 -- 172.21.15.99:0/1646315196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632e137b620 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.641+0000 7faed27fc640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e163fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faedaffd640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 0x5632e163f7a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faecc00bea0 tx=0x7faecc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113e8df8febbc851 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faec27fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faec27fc640 1 -- 172.21.15.99:0/1646315196 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc01f060 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faec27fc640 1 -- 172.21.15.99:0/1646315196 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7faecc0108a0 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec40054d0 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faec27fc640 1 -- 172.21.15.99:0/1646315196 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faecc010aa0 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 10 client.16368 fetch_fsmap learned FSMap version 35 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5632e16512a0 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faec27fc640 1 -- 172.21.15.99:0/1646315196 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec40054d0 con 0x5632e1638210 2024-05-14T17:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faec27fc640 1 -- 172.21.15.99:0/1646315196 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7faecc019030 con 0x5632e1638210 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 10 client.16368 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 20 client.16368 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e1652800 0x5632e1654c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 4 client.16368 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.642+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632e15acb80 con 0x5632e1652800 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.643+0000 7faed27fc640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e1652800 0x5632e1654c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.643+0000 7faec27fc640 10 client.16368 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.643+0000 7faed27fc640 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e1652800 0x5632e1654c50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da94291b7fe06525 server_cookie=a75a0a6ff3610d3a in_seq=0 out_seq=0 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faec27fc640 1 -- 172.21.15.99:0/1646315196 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632e15acb80 con 0x5632e1652800 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faec27fc640 10 client.16368 handle_command_reply: tid=0 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 client.16368 shutdown 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 msgr2=0x5632e163f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 0x5632e163f7a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faecc00bea0 tx=0x7faecc00bf90 comp rx=0 tx=0).stop 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 shutdown_connections 2024-05-14T17:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e1652800 0x5632e1654c50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e163fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e1638210 0x5632e163f7a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faec27fc640 0 client.16368 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 --2- 172.21.15.99:0/1646315196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e16378f0 0x5632e16424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 >> 172.21.15.99:0/1646315196 conn(0x5632e1633e00 msgr2=0x5632e1633100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 shutdown_connections 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 1 -- 172.21.15.99:0/1646315196 wait complete. 2024-05-14T17:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.644+0000 7faee4410740 20 client.16368 trim_cache size 0 max 16384 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.659+0000 7faee4410740 1 Processor -- start 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.659+0000 7faee4410740 1 -- start start 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faee4410740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e165a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faee4410740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e165aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faee4410740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1642ad0 0x5632e163ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faee4410740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faedc1b0ed0 con 0x5632e16378f0 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faee4410740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faedc1ae770 con 0x5632e1638210 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faee4410740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faedc1ac5e0 con 0x5632e1642ad0 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e165a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e165a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52696/0 (socket says 172.21.15.99:52696) 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 -- 172.21.15.99:0/2904587512 learned_addr learned my addr 172.21.15.99:0/2904587512 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faedaffd640 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e165aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 -- 172.21.15.99:0/2904587512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1642ad0 msgr2=0x5632e163ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1642ad0 0x5632e163ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 -- 172.21.15.99:0/2904587512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 msgr2=0x5632e165aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e165aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 -- 172.21.15.99:0/2904587512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632e1644ec0 con 0x5632e16378f0 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed27fc640 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e165a520 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7faec4004880 tx=0x7faec400a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a364ebea5fe7a46e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed0ff9640 1 -- 172.21.15.99:0/2904587512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400fa50 con 0x5632e16378f0 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed0ff9640 1 -- 172.21.15.99:0/2904587512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec400fbf0 con 0x5632e16378f0 2024-05-14T17:12:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.660+0000 7faed0ff9640 1 -- 172.21.15.99:0/2904587512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec401e470 con 0x5632e16378f0 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 -- 172.21.15.99:0/2904587512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 msgr2=0x5632e165a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e165a520 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7faec4004880 tx=0x7faec400a640 comp rx=0 tx=0).stop 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 -- 172.21.15.99:0/2904587512 shutdown_connections 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1642ad0 0x5632e163ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1638210 0x5632e165aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 --2- 172.21.15.99:0/2904587512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e165a520 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 -- 172.21.15.99:0/2904587512 >> 172.21.15.99:0/2904587512 conn(0x5632e1633e00 msgr2=0x5632e164d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 -- 172.21.15.99:0/2904587512 shutdown_connections 2024-05-14T17:12:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.661+0000 7faee4410740 1 -- 172.21.15.99:0/2904587512 wait complete. 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 -- start start 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 Processor -- start 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e164b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1638210 0x5632e1648470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e1648b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faedc1acc30 con 0x5632e16378f0 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faedc1ae0a0 con 0x5632e1642ad0 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faedc1ad300 con 0x5632e1638210 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faee4410740 10 client.0 _ll_register_callbacks cb 0x7faecc00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e164b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e164b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52704/0 (socket says 172.21.15.99:52704) 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 -- 172.21.15.99:0/528148559 learned_addr learned my addr 172.21.15.99:0/528148559 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faedaffd640 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1638210 0x5632e1648470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 -- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1638210 msgr2=0x5632e1648470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1638210 0x5632e1648470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 -- 172.21.15.99:0/528148559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 msgr2=0x5632e1648b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e1648b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 -- 172.21.15.99:0/528148559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632e16bc0e0 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faed27fc640 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e164b0e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7faec4011680 tx=0x7faec4029a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1bd18bf2468a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.662+0000 7faec27fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faec27fc640 1 -- 172.21.15.99:0/528148559 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4036060 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faec27fc640 1 -- 172.21.15.99:0/528148559 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7faec400f8c0 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec400fbf0 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faec27fc640 1 -- 172.21.15.99:0/528148559 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faec4029340 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 10 client.16374 fetch_fsmap learned FSMap version 35 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5632e16bd290 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faec27fc640 1 -- 172.21.15.99:0/528148559 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec400fbf0 con 0x5632e16378f0 2024-05-14T17:12:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faec27fc640 1 -- 172.21.15.99:0/528148559 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7faec401eb30 con 0x5632e16378f0 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 10 client.16374 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 20 client.16374 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e16bd710 0x5632e16bfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 4 client.16374 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.663+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632e15a6360 con 0x5632e16bd710 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.664+0000 7faedaffd640 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e16bd710 0x5632e16bfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.664+0000 7faec27fc640 10 client.16374 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:36.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.664+0000 7faedaffd640 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e16bd710 0x5632e16bfb00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcc23a96745e404 server_cookie=11b878e3052f6e72 in_seq=0 out_seq=0 2024-05-14T17:12:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.705+0000 7faec27fc640 1 -- 172.21.15.99:0/528148559 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5632e15a6360 con 0x5632e16bd710 2024-05-14T17:12:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.705+0000 7faec27fc640 10 client.16374 handle_command_reply: tid=0 2024-05-14T17:12:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.705+0000 7faee4410740 1 client.16374 shutdown 2024-05-14T17:12:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.705+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 msgr2=0x5632e164b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.705+0000 7faee4410740 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e164b0e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7faec4011680 tx=0x7faec4029a40 comp rx=0 tx=0).stop 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 shutdown_connections 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5632e16bd710 0x5632e16bfb00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632e1642ad0 0x5632e1648b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632e1638210 0x5632e1648470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faec27fc640 0 client.16374 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 --2- 172.21.15.99:0/528148559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632e16378f0 0x5632e164b0e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 >> 172.21.15.99:0/528148559 conn(0x5632e1633e00 msgr2=0x5632e164def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 shutdown_connections 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 1 -- 172.21.15.99:0/528148559 wait complete. 2024-05-14T17:12:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee4410740 20 client.16374 trim_cache size 0 max 16384 2024-05-14T17:12:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:12:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:12:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:12:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faec80b2a10 msgr2=0x7faec80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faec80b2a10 0x7faec80b4ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faed40c6b10 tx=0x7faed40c6a80 comp rx=0 tx=0).stop 2024-05-14T17:12:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 msgr2=0x7faedc1a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 0x7faedc1a5040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faed4005cc0 tx=0x7faed4003c20 comp rx=0 tx=0).stop 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 shutdown_connections 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7faec80b2a10 0x7faec80b4ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faedc107960 0x7faedc1a5040 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faedc102310 0x7faedc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 --2- 172.21.15.99:0/1116037773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faedc101950 0x7faedc199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.706+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 >> 172.21.15.99:0/1116037773 conn(0x7faedc078a60 msgr2=0x7faedc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.707+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 shutdown_connections 2024-05-14T17:12:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:36.707+0000 7faee16f8640 1 -- 172.21.15.99:0/1116037773 wait complete. 2024-05-14T17:12:37.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:36 smithi150 ceph-mon[36204]: pgmap v280: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 619 KiB/s wr, 76 op/s 2024-05-14T17:12:37.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:36 smithi099 ceph-mon[31851]: pgmap v280: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 619 KiB/s wr, 76 op/s 2024-05-14T17:12:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:36 smithi193 ceph-mon[35962]: pgmap v280: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 619 KiB/s wr, 76 op/s 2024-05-14T17:12:37.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 395026 4.77 MB/sec execute 289 sec latency 1489.829 ms 2024-05-14T17:12:38.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 395459 4.76 MB/sec execute 290 sec latency 783.687 ms 2024-05-14T17:12:39.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:38 smithi150 ceph-mon[36204]: pgmap v281: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 619 KiB/s wr, 76 op/s 2024-05-14T17:12:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:38 smithi099 ceph-mon[31851]: pgmap v281: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 619 KiB/s wr, 76 op/s 2024-05-14T17:12:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:38 smithi193 ceph-mon[35962]: pgmap v281: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 619 KiB/s wr, 76 op/s 2024-05-14T17:12:39.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 395459 4.75 MB/sec execute 291 sec latency 1783.785 ms 2024-05-14T17:12:40.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 397781 4.78 MB/sec execute 292 sec latency 1806.811 ms 2024-05-14T17:12:41.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:40 smithi150 ceph-mon[36204]: pgmap v282: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 958 KiB/s wr, 129 op/s 2024-05-14T17:12:41.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:40 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:40 smithi099 ceph-mon[31851]: pgmap v282: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 958 KiB/s wr, 129 op/s 2024-05-14T17:12:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:40 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:40 smithi193 ceph-mon[35962]: pgmap v282: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 958 KiB/s wr, 129 op/s 2024-05-14T17:12:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:40 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:41.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 399001 4.78 MB/sec execute 293 sec latency 233.752 ms 2024-05-14T17:12:42.543 INFO:tasks.workunit.client.0.smithi099.stdout: 1 399001 4.76 MB/sec execute 294 sec latency 1233.833 ms 2024-05-14T17:12:43.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:42 smithi150 ceph-mon[36204]: pgmap v283: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 650 KiB/s wr, 91 op/s 2024-05-14T17:12:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:42 smithi099 ceph-mon[31851]: pgmap v283: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 650 KiB/s wr, 91 op/s 2024-05-14T17:12:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:42 smithi193 ceph-mon[35962]: pgmap v283: 97 pgs: 97 active+clean; 102 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 650 KiB/s wr, 91 op/s 2024-05-14T17:12:43.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 399001 4.75 MB/sec execute 295 sec latency 2233.960 ms 2024-05-14T17:12:44.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 399001 4.73 MB/sec execute 296 sec latency 3234.087 ms 2024-05-14T17:12:45.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:44 smithi150 ceph-mon[36204]: pgmap v284: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 950 KiB/s wr, 126 op/s 2024-05-14T17:12:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:44 smithi099 ceph-mon[31851]: pgmap v284: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 950 KiB/s wr, 126 op/s 2024-05-14T17:12:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:44 smithi193 ceph-mon[35962]: pgmap v284: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 950 KiB/s wr, 126 op/s 2024-05-14T17:12:45.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 401326 4.76 MB/sec execute 297 sec latency 3261.645 ms 2024-05-14T17:12:46.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.76 MB/sec execute 298 sec latency 379.966 ms 2024-05-14T17:12:47.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:46 smithi150 ceph-mon[36204]: pgmap v285: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 88 op/s 2024-05-14T17:12:47.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:46 smithi099 ceph-mon[31851]: pgmap v285: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 88 op/s 2024-05-14T17:12:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:46 smithi193 ceph-mon[35962]: pgmap v285: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 88 op/s 2024-05-14T17:12:47.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.75 MB/sec execute 299 sec latency 1380.068 ms 2024-05-14T17:12:48.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.73 MB/sec execute 300 sec latency 2380.183 ms 2024-05-14T17:12:49.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:48 smithi150 ceph-mon[36204]: pgmap v286: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 88 op/s 2024-05-14T17:12:49.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:48 smithi099 ceph-mon[31851]: pgmap v286: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 88 op/s 2024-05-14T17:12:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:48 smithi193 ceph-mon[35962]: pgmap v286: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 88 op/s 2024-05-14T17:12:49.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 402551 4.72 MB/sec execute 301 sec latency 3380.225 ms 2024-05-14T17:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 -- 172.21.15.99:0/638369073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 msgr2=0x7f8cbc10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 --2- 172.21.15.99:0/638369073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc10e790 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009ca0 tx=0x7f8cb402a8c0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 -- 172.21.15.99:0/638369073 shutdown_connections 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 --2- 172.21.15.99:0/638369073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc10e790 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 --2- 172.21.15.99:0/638369073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cbc101bc0 0x7f8cbc102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 --2- 172.21.15.99:0/638369073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cbc1079f0 0x7f8cbc107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 -- 172.21.15.99:0/638369073 >> 172.21.15.99:0/638369073 conn(0x7f8cbc078830 msgr2=0x7f8cbc078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 -- 172.21.15.99:0/638369073 shutdown_connections 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 -- 172.21.15.99:0/638369073 wait complete. 2024-05-14T17:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.809+0000 7f8cc15af640 1 Processor -- start 2024-05-14T17:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 -- start start 2024-05-14T17:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cbc101bc0 0x7f8cbc19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cbc1079f0 0x7f8cbc1a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111c20 con 0x7f8cbc102560 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111aa0 con 0x7f8cbc101bc0 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cc15af640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111da0 con 0x7f8cbc1079f0 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43818/0 (socket says 172.21.15.99:43818) 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 -- 172.21.15.99:0/1705687071 learned_addr learned my addr 172.21.15.99:0/1705687071 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cbc101bc0 0x7f8cbc19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cbc1079f0 0x7f8cbc1a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 -- 172.21.15.99:0/1705687071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cbc1079f0 msgr2=0x7f8cbc1a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cbc1079f0 0x7f8cbc1a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 -- 172.21.15.99:0/1705687071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cbc101bc0 msgr2=0x7f8cbc19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cbc101bc0 0x7f8cbc19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 -- 172.21.15.99:0/1705687071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc1a55c0 con 0x7f8cbc102560 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.810+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc19a540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.811+0000 7f8cb8ff9640 1 -- 172.21.15.99:0/1705687071 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400cd80 con 0x7f8cbc102560 2024-05-14T17:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.811+0000 7f8cb8ff9640 1 -- 172.21.15.99:0/1705687071 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca40054d0 con 0x7f8cbc102560 2024-05-14T17:12:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.811+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc1a5790 con 0x7f8cbc102560 2024-05-14T17:12:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.811+0000 7f8cb8ff9640 1 -- 172.21.15.99:0/1705687071 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4010690 con 0x7f8cbc102560 2024-05-14T17:12:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.811+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a5c60 con 0x7f8cbc102560 2024-05-14T17:12:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.813+0000 7f8cb8ff9640 1 -- 172.21.15.99:0/1705687071 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ca4004e90 con 0x7f8cbc102560 2024-05-14T17:12:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.813+0000 7f8cc43a3740 1 Processor -- start 2024-05-14T17:12:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 -- start start 2024-05-14T17:12:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c73ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c749590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c74bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56439c6b1c30 con 0x56439c73f210 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56439c6a17d0 con 0x56439c749ad0 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56439c6a5810 con 0x56439c73e8f0 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c749590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c749590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43822/0 (socket says 172.21.15.99:43822) 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbaffd640 1 -- 172.21.15.99:0/1047152091 learned_addr learned my addr 172.21.15.99:0/1047152091 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c73ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c74bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cb8ff9640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8ca80b2a10 0x7f8ca80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 -- 172.21.15.99:0/1047152091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 msgr2=0x56439c74bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c74bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 -- 172.21.15.99:0/1047152091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 msgr2=0x56439c749590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c749590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 -- 172.21.15.99:0/1047152091 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439c3b6210 con 0x56439c73e8f0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c749590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.814+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8ca80b2a10 0x7f8ca80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.815+0000 7f8cb8ff9640 1 -- 172.21.15.99:0/1705687071 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f8ca40e2a10 con 0x7f8cbc102560 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.815+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c73ecd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4005e30 tx=0x7f8cb4005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5161dc209a29d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.815+0000 7f8cb17fa640 1 -- 172.21.15.99:0/1047152091 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4042040 con 0x56439c73e8f0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.815+0000 7f8cb17fa640 1 -- 172.21.15.99:0/1047152091 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb4036070 con 0x56439c73e8f0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.815+0000 7f8cb17fa640 1 -- 172.21.15.99:0/1047152091 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4004600 con 0x56439c73e8f0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- 172.21.15.99:0/1047152091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 msgr2=0x56439c73ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c73ecd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4005e30 tx=0x7f8cb4005fa0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8ca80b2a10 0x7f8ca80b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4049000 tx=0x7f8cb404e000 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- 172.21.15.99:0/1047152091 shutdown_connections 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c74bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c749590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/1047152091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c73ecd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- 172.21.15.99:0/1047152091 >> 172.21.15.99:0/1047152091 conn(0x56439c73ae00 msgr2=0x56439c73b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- 172.21.15.99:0/1047152091 shutdown_connections 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- 172.21.15.99:0/1047152091 wait complete. 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- start start 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 Processor -- start 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 0x56439c749420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73f210 0x56439c746710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c746c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56439c387790 con 0x56439c73e8f0 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56439c65d890 con 0x56439c73f210 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56439c74c7a0 con 0x56439c749ad0 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.816+0000 7f8cc43a3740 10 client.0 _ll_register_callbacks cb 0x56439c660d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 0x56439c749420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 0x56439c749420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43830/0 (socket says 172.21.15.99:43830) 2024-05-14T17:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c746c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3283037021 learned_addr learned my addr 172.21.15.99:0/3283037021 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73f210 0x56439c746710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 msgr2=0x56439c746c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c746c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3283037021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73f210 msgr2=0x56439c746710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73f210 0x56439c746710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3283037021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439c482620 con 0x56439c73e8f0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 0x56439c749420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0126e0 tx=0x7f8cac012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8880b78b532386bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8ca27fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8ca27fc640 1 -- 172.21.15.99:0/3283037021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac004050 con 0x56439c73e8f0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8ca27fc640 1 -- 172.21.15.99:0/3283037021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f8cac0041f0 con 0x56439c73e8f0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.817+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb4036070 con 0x56439c73e8f0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8ca27fc640 1 -- 172.21.15.99:0/3283037021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8cac0043f0 con 0x56439c73e8f0 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 10 client.16386 fetch_fsmap learned FSMap version 36 2024-05-14T17:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56439c758220 con 0x56439c73e8f0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8ca27fc640 1 -- 172.21.15.99:0/3283037021 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb4036070 con 0x56439c73e8f0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8ca27fc640 1 -- 172.21.15.99:0/3283037021 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8cac0037f0 con 0x56439c73e8f0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 10 client.16386 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 20 client.16386 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7596e0 0x56439c75baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 4 client.16386 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56439c6b3b80 con 0x56439c7596e0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.818+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7596e0 0x56439c75baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.819+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7596e0 0x56439c75baf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729b2ad407c69b15 server_cookie=3e9b9b2b3962e098 in_seq=0 out_seq=0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.819+0000 7f8ca27fc640 10 client.16386 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.819+0000 7f8ca27fc640 1 -- 172.21.15.99:0/3283037021 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56439c6b3b80 con 0x56439c7596e0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.819+0000 7f8ca27fc640 10 client.16386 handle_command_reply: tid=0 2024-05-14T17:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.819+0000 7f8cc43a3740 1 client.16386 shutdown 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 msgr2=0x56439c749420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 0x56439c749420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0126e0 tx=0x7f8cac012ae0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 shutdown_connections 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7596e0 0x56439c75baf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c746c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73f210 0x56439c746710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8ca27fc640 0 client.16386 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3283037021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73e8f0 0x56439c749420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 >> 172.21.15.99:0/3283037021 conn(0x56439c73ae00 msgr2=0x56439c73a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 shutdown_connections 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3283037021 wait complete. 2024-05-14T17:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.820+0000 7f8cc43a3740 20 client.16386 trim_cache size 0 max 16384 2024-05-14T17:12:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 Processor -- start 2024-05-14T17:12:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 -- start start 2024-05-14T17:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73e8f0 0x56439c741dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c748f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c7409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1b0e80 con 0x56439c73f210 2024-05-14T17:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ae6f0 con 0x56439c73e8f0 2024-05-14T17:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ac560 con 0x56439c749ad0 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cb27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c748f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73e8f0 0x56439c741dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cb27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c748f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43846/0 (socket says 172.21.15.99:43846) 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cb27fc640 1 -- 172.21.15.99:0/3740250405 learned_addr learned my addr 172.21.15.99:0/3740250405 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.836+0000 7f8cbb7fe640 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c7409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3740250405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 msgr2=0x56439c7409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c7409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3740250405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 msgr2=0x56439c748f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c748f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cbaffd640 1 -- 172.21.15.99:0/3740250405 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439c73ac30 con 0x56439c73e8f0 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c748f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73e8f0 0x56439c741dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8cac003dd0 tx=0x7f8cac0105c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421937d202dbe31a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cb0ff9640 1 -- 172.21.15.99:0/3740250405 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac029040 con 0x56439c73e8f0 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cb0ff9640 1 -- 172.21.15.99:0/3740250405 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cac003e50 con 0x56439c73e8f0 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.837+0000 7f8cb0ff9640 1 -- 172.21.15.99:0/3740250405 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac005570 con 0x56439c73e8f0 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3740250405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73e8f0 msgr2=0x56439c741dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73e8f0 0x56439c741dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8cac003dd0 tx=0x7f8cac0105c0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3740250405 shutdown_connections 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c749ad0 0x56439c7409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c748f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/3740250405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c73e8f0 0x56439c741dc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3740250405 >> 172.21.15.99:0/3740250405 conn(0x56439c73ae00 msgr2=0x56439c7402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3740250405 shutdown_connections 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 -- 172.21.15.99:0/3740250405 wait complete. 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 -- start start 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 Processor -- start 2024-05-14T17:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c761200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.838+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c761740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cc43a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c7bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1acbb0 con 0x56439c73f210 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ae020 con 0x56439c749ad0 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cc43a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ad280 con 0x56439c73e8f0 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cc43a3740 10 client.0 _ll_register_callbacks cb 0x7f8cac009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cb27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c761740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c761200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cb27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c761740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43858/0 (socket says 172.21.15.99:43858) 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cb27fc640 1 -- 172.21.15.99:0/580772699 learned_addr learned my addr 172.21.15.99:0/580772699 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c7bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 -- 172.21.15.99:0/580772699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 msgr2=0x56439c7bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c7bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 -- 172.21.15.99:0/580772699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 msgr2=0x56439c761740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c761740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 -- 172.21.15.99:0/580772699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439c7bfd70 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c761740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cbaffd640 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c761200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8cac003be0 tx=0x7f8cac00f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d7f101eebfbdc0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8ca27fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8ca27fc640 1 -- 172.21.15.99:0/580772699 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac029040 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cac003e50 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.839+0000 7f8ca27fc640 1 -- 172.21.15.99:0/580772699 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f8cac0038c0 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8ca27fc640 1 -- 172.21.15.99:0/580772699 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8cac0052b0 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 10 client.25807 fetch_fsmap learned FSMap version 36 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56439c7c8320 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8ca27fc640 1 -- 172.21.15.99:0/580772699 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cac003e50 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8ca27fc640 1 -- 172.21.15.99:0/580772699 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8cac033020 con 0x56439c73e8f0 2024-05-14T17:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 10 client.25807 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 10 client.25807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 20 client.25807 populate_metadata read hostname 'smithi099' 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7c89d0 0x56439c7cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 4 client.25807 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56439c6ad360 con 0x56439c7c89d0 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.840+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7c89d0 0x56439c7cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.841+0000 7f8cb27fc640 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7c89d0 0x56439c7cadc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ef154e67045cc6 server_cookie=1cfbc6ed37936889 in_seq=0 out_seq=0 2024-05-14T17:12:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.841+0000 7f8ca27fc640 10 client.25807 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8ca27fc640 1 -- 172.21.15.99:0/580772699 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56439c6ad360 con 0x56439c7c89d0 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8ca27fc640 10 client.25807 handle_command_reply: tid=0 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 client.25807 shutdown 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 msgr2=0x56439c761200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c761200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8cac003be0 tx=0x7f8cac00f450 comp rx=0 tx=0).stop 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 shutdown_connections 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56439c7c89d0 0x56439c7cadc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56439c749ad0 0x56439c7bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8ca27fc640 0 client.25807 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56439c73f210 0x56439c761740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 --2- 172.21.15.99:0/580772699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56439c73e8f0 0x56439c761200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 >> 172.21.15.99:0/580772699 conn(0x56439c73ae00 msgr2=0x56439c741480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 shutdown_connections 2024-05-14T17:12:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 1 -- 172.21.15.99:0/580772699 wait complete. 2024-05-14T17:12:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.876+0000 7f8cc43a3740 20 client.25807 trim_cache size 0 max 16384 2024-05-14T17:12:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:12:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:12:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8ca80b2a10 msgr2=0x7f8ca80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8ca80b2a10 0x7f8ca80b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4049000 tx=0x7f8cb404e000 comp rx=0 tx=0).stop 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 msgr2=0x7f8cbc19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc19a540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400edf0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 shutdown_connections 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f8ca80b2a10 0x7f8ca80b4ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cbc1079f0 0x7f8cbc1a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102560 0x7f8cbc19a540 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 --2- 172.21.15.99:0/1705687071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cbc101bc0 0x7f8cbc19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 >> 172.21.15.99:0/1705687071 conn(0x7f8cbc078830 msgr2=0x7f8cbc1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 shutdown_connections 2024-05-14T17:12:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:12:49.877+0000 7f8cc15af640 1 -- 172.21.15.99:0/1705687071 wait complete. 2024-05-14T17:12:50.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 404823 4.75 MB/sec execute 302 sec latency 3407.939 ms 2024-05-14T17:12:51.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:50 smithi150 ceph-mon[36204]: pgmap v287: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-14T17:12:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:50 smithi099 ceph-mon[31851]: pgmap v287: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-14T17:12:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:50 smithi193 ceph-mon[35962]: pgmap v287: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-05-14T17:12:51.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.75 MB/sec execute 303 sec latency 294.893 ms 2024-05-14T17:12:52.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.73 MB/sec execute 304 sec latency 1294.969 ms 2024-05-14T17:12:53.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:52 smithi150 ceph-mon[36204]: pgmap v288: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 928 KiB/s wr, 80 op/s 2024-05-14T17:12:53.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:52 smithi099 ceph-mon[31851]: pgmap v288: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 928 KiB/s wr, 80 op/s 2024-05-14T17:12:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:52 smithi193 ceph-mon[35962]: pgmap v288: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 928 KiB/s wr, 80 op/s 2024-05-14T17:12:53.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.72 MB/sec execute 305 sec latency 2295.067 ms 2024-05-14T17:12:54.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 406092 4.70 MB/sec execute 306 sec latency 3295.147 ms 2024-05-14T17:12:55.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:54 smithi150 ceph-mon[36204]: pgmap v289: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-14T17:12:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:54 smithi099 ceph-mon[31851]: pgmap v289: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-14T17:12:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:54 smithi193 ceph-mon[35962]: pgmap v289: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-05-14T17:12:55.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 408483 4.73 MB/sec execute 307 sec latency 3324.577 ms 2024-05-14T17:12:56.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:55 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:12:56.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:55 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:12:56.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:55 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:12:56.544 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.73 MB/sec execute 308 sec latency 319.248 ms 2024-05-14T17:12:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:56 smithi099 ceph-mon[31851]: pgmap v290: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T17:12:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:56 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:12:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:56 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:12:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:56 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:12:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:56 smithi193 ceph-mon[35962]: pgmap v290: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T17:12:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:56 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:12:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:56 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:12:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:56 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:12:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:56 smithi150 ceph-mon[36204]: pgmap v290: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T17:12:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:56 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:12:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:56 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:12:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:56 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:12:57.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.72 MB/sec execute 309 sec latency 1319.287 ms 2024-05-14T17:12:58.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.70 MB/sec execute 310 sec latency 2319.336 ms 2024-05-14T17:12:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:12:58 smithi099 ceph-mon[31851]: pgmap v291: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T17:12:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:12:58 smithi193 ceph-mon[35962]: pgmap v291: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T17:12:59.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:12:58 smithi150 ceph-mon[36204]: pgmap v291: 97 pgs: 97 active+clean; 107 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-05-14T17:12:59.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 409641 4.69 MB/sec execute 311 sec latency 3319.415 ms 2024-05-14T17:13:00.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 411933 4.72 MB/sec execute 312 sec latency 3350.953 ms 2024-05-14T17:13:01.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:00 smithi099 ceph-mon[31851]: pgmap v292: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-05-14T17:13:01.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:00 smithi193 ceph-mon[35962]: pgmap v292: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-05-14T17:13:01.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:00 smithi150 ceph-mon[36204]: pgmap v292: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-05-14T17:13:01.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413150 4.72 MB/sec execute 313 sec latency 33.511 ms 2024-05-14T17:13:02.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 413183 4.70 MB/sec execute 314 sec latency 983.881 ms 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 -- 172.21.15.99:0/2794179389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 msgr2=0x7f009c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/2794179389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c101e00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f008c009cd0 tx=0x7f008c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 -- 172.21.15.99:0/2794179389 shutdown_connections 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/2794179389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009c102340 0x7f009c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/2794179389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c101e00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/2794179389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f009c107940 0x7f009c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 -- 172.21.15.99:0/2794179389 >> 172.21.15.99:0/2794179389 conn(0x7f009c078ad0 msgr2=0x7f009c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 -- 172.21.15.99:0/2794179389 shutdown_connections 2024-05-14T17:13:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.974+0000 7f00a2bb6640 1 -- 172.21.15.99:0/2794179389 wait complete. 2024-05-14T17:13:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 Processor -- start 2024-05-14T17:13:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 -- start start 2024-05-14T17:13:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009c102340 0x7f009c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f009c107940 0x7f009c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f009c111c00 con 0x7f009c101980 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f009c111a80 con 0x7f009c107940 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a2bb6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f009c111d80 con 0x7f009c102340 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a0952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a0952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39312/0 (socket says 172.21.15.99:39312) 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a0952640 1 -- 172.21.15.99:0/672701878 learned_addr learned my addr 172.21.15.99:0/672701878 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f00a1153640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f009c107940 0x7f009c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.975+0000 7f009bfff640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009c102340 0x7f009c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a0952640 1 -- 172.21.15.99:0/672701878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009c102340 msgr2=0x7f009c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a0952640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009c102340 0x7f009c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a0952640 1 -- 172.21.15.99:0/672701878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f009c107940 msgr2=0x7f009c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a0952640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f009c107940 0x7f009c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a0952640 1 -- 172.21.15.99:0/672701878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c19d030 con 0x7f009c101980 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a0952640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c198040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0094007b50 tx=0x7f009400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f0099ffb640 1 -- 172.21.15.99:0/672701878 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094007c40 con 0x7f009c101980 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f0099ffb640 1 -- 172.21.15.99:0/672701878 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0094007de0 con 0x7f009c101980 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c19d320 con 0x7f009c101980 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f0099ffb640 1 -- 172.21.15.99:0/672701878 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009400ace0 con 0x7f009c101980 2024-05-14T17:13:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.976+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c06a100 con 0x7f009c101980 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f0099ffb640 1 -- 172.21.15.99:0/672701878 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f009400eb30 con 0x7f009c101980 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 Processor -- start 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 -- start start 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94064cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 0x560c9406f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c9406fad0 0x560c94071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560c93fd7c30 con 0x560c94065210 2024-05-14T17:13:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560c93fc77d0 con 0x560c940648f0 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.979+0000 7f00a59a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c93fcb810 con 0x560c9406fad0 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 0x560c9406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f0099ffb640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00880b2a90 0x7f00880b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 0x560c9406f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39328/0 (socket says 172.21.15.99:39328) 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f009bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c9406fad0 0x560c94071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 -- 172.21.15.99:0/2714807936 learned_addr learned my addr 172.21.15.99:0/2714807936 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a1153640 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94064cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a1153640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00880b2a90 0x7f00880b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f0099ffb640 1 -- 172.21.15.99:0/672701878 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f00940e3bc0 con 0x7f009c101980 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 -- 172.21.15.99:0/2714807936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c9406fad0 msgr2=0x560c94071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c9406fad0 0x560c94071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 -- 172.21.15.99:0/2714807936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 msgr2=0x560c94064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94064cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 -- 172.21.15.99:0/2714807936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c93cdc210 con 0x560c94065210 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f009bfff640 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c9406fad0 0x560c94071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a0952640 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 0x560c9406f590 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f00940822f0 tx=0x7f0094000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b123477e42231f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f00a1153640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00880b2a90 0x7f00880b4f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0084009fd0 tx=0x7f0084005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f0092ffd640 1 -- 172.21.15.99:0/2714807936 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940ab630 con 0x560c94065210 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f0092ffd640 1 -- 172.21.15.99:0/2714807936 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f009400ee20 con 0x560c94065210 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.980+0000 7f0092ffd640 1 -- 172.21.15.99:0/2714807936 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940a71c0 con 0x560c94065210 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 -- 172.21.15.99:0/2714807936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 msgr2=0x560c9406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 0x560c9406f590 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f00940822f0 tx=0x7f0094000bc0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 -- 172.21.15.99:0/2714807936 shutdown_connections 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c9406fad0 0x560c94071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c94065210 0x560c9406f590 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2714807936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94064cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 -- 172.21.15.99:0/2714807936 >> 172.21.15.99:0/2714807936 conn(0x560c94060e00 msgr2=0x560c940611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 -- 172.21.15.99:0/2714807936 shutdown_connections 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 -- 172.21.15.99:0/2714807936 wait complete. 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.981+0000 7f00a59a9740 1 -- start start 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 Processor -- start 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c9406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c9406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560c93cad790 con 0x560c9406fad0 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560c93f83890 con 0x560c940648f0 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c940727a0 con 0x560c94065210 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a59a9740 10 client.0 _ll_register_callbacks cb 0x560c93f86d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c9406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c9406cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39332/0 (socket says 172.21.15.99:39332) 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 -- 172.21.15.99:0/4009898724 learned_addr learned my addr 172.21.15.99:0/4009898724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a0952640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c9406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f009bfff640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 -- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 msgr2=0x560c9406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 -- 172.21.15.99:0/4009898724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 msgr2=0x560c9406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c9406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a1153640 1 -- 172.21.15.99:0/4009898724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c93da8620 con 0x560c9406fad0 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f00a0952640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c9406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.982+0000 7f009bfff640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a1153640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c9406cdc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f008400df40 tx=0x7f00840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b20b8bbf605574 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f007bfff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f007bfff640 1 -- 172.21.15.99:0/4009898724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008400fe60 con 0x560c9406fad0 2024-05-14T17:13:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f007bfff640 1 -- 172.21.15.99:0/4009898724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f0084012030 con 0x560c9406fad0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f009400ee20 con 0x560c9406fad0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f007bfff640 1 -- 172.21.15.99:0/4009898724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0084012230 con 0x560c9406fad0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 10 client.16404 fetch_fsmap learned FSMap version 36 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560c9407e2f0 con 0x560c9406fad0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f007bfff640 1 -- 172.21.15.99:0/4009898724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f009400ee20 con 0x560c9406fad0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f007bfff640 1 -- 172.21.15.99:0/4009898724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f0084027070 con 0x560c9406fad0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 10 client.16404 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 20 client.16404 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c9407f800 0x560c94081bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 4 client.16404 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.983+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c93fd9b80 con 0x560c9407f800 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.984+0000 7f00a0952640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c9407f800 0x560c94081bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.984+0000 7f007bfff640 10 client.16404 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.984+0000 7f00a0952640 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c9407f800 0x560c94081bf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33a21b5ee115a2d server_cookie=51bf40a47f5efc8 in_seq=0 out_seq=0 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f007bfff640 1 -- 172.21.15.99:0/4009898724 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c93fd9b80 con 0x560c9407f800 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f007bfff640 10 client.16404 handle_command_reply: tid=0 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 client.16404 shutdown 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 msgr2=0x560c9406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c9406cdc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f008400df40 tx=0x7f00840046c0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 shutdown_connections 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c9407f800 0x560c94081bf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c9406cdc0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 --2- 172.21.15.99:0/4009898724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c9406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f007bfff640 0 client.16404 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 >> 172.21.15.99:0/4009898724 conn(0x560c94060e00 msgr2=0x560c94060100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 shutdown_connections 2024-05-14T17:13:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 1 -- 172.21.15.99:0/4009898724 wait complete. 2024-05-14T17:13:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:02.985+0000 7f00a59a9740 20 client.16404 trim_cache size 0 max 16384 2024-05-14T17:13:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.000+0000 7f00a59a9740 1 Processor -- start 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.000+0000 7f00a59a9740 1 -- start start 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 0x560c9407b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c9406fad0 0x560c94087500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a59a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1aecc0 con 0x560c940648f0 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a59a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1ac530 con 0x560c9406fad0 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a59a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1aa3a0 con 0x560c94065210 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 0x560c9407b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 0x560c9407b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39348/0 (socket says 172.21.15.99:39348) 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 -- 172.21.15.99:0/2914121293 learned_addr learned my addr 172.21.15.99:0/2914121293 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f009bfff640 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 -- 172.21.15.99:0/2914121293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 msgr2=0x560c9406eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 -- 172.21.15.99:0/2914121293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c9406fad0 msgr2=0x560c94087500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c9406fad0 0x560c94087500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 -- 172.21.15.99:0/2914121293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c9405c830 con 0x560c940648f0 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00a0952640 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 0x560c9407b670 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f00940e13c0 tx=0x7f009400a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36467a934119a41c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00927fc640 1 -- 172.21.15.99:0/2914121293 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009400ee20 con 0x560c940648f0 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00927fc640 1 -- 172.21.15.99:0/2914121293 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00940ab630 con 0x560c940648f0 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.001+0000 7f00927fc640 1 -- 172.21.15.99:0/2914121293 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940b3ad0 con 0x560c940648f0 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 -- 172.21.15.99:0/2914121293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 msgr2=0x560c9407b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 0x560c9407b670 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f00940e13c0 tx=0x7f009400a700 comp rx=0 tx=0).stop 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 -- 172.21.15.99:0/2914121293 shutdown_connections 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c9406fad0 0x560c94087500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c9406eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 --2- 172.21.15.99:0/2914121293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c940648f0 0x560c9407b670 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 -- 172.21.15.99:0/2914121293 >> 172.21.15.99:0/2914121293 conn(0x560c94060e00 msgr2=0x560c94066b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 -- 172.21.15.99:0/2914121293 shutdown_connections 2024-05-14T17:13:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 -- 172.21.15.99:0/2914121293 wait complete. 2024-05-14T17:13:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 -- start start 2024-05-14T17:13:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.002+0000 7f00a59a9740 1 Processor -- start 2024-05-14T17:13:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94077e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c94073220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c94073760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1aa9f0 con 0x560c9406fad0 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1abe60 con 0x560c940648f0 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1ab0c0 con 0x560c94065210 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 10 client.0 _ll_register_callbacks cb 0x7f008400daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c94073760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c94073760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39360/0 (socket says 172.21.15.99:39360) 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 -- 172.21.15.99:0/479822089 learned_addr learned my addr 172.21.15.99:0/479822089 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f009bfff640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c94073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 -- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 msgr2=0x560c94073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c94073220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a0952640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94077e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 -- 172.21.15.99:0/479822089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 msgr2=0x560c94077e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94077e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 -- 172.21.15.99:0/479822089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c94073ea0 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f009bfff640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c94073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a1153640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c94073760 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0084004de0 tx=0x7f0084012640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a1efd4768b71da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f007bfff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f007bfff640 1 -- 172.21.15.99:0/479822089 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084005440 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00940ab630 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.003+0000 7f007bfff640 1 -- 172.21.15.99:0/479822089 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f00840055e0 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f007bfff640 1 -- 172.21.15.99:0/479822089 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f008403f050 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 10 client.16410 fetch_fsmap learned FSMap version 36 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560c940e9ff0 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f007bfff640 1 -- 172.21.15.99:0/479822089 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00940ab630 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f007bfff640 1 -- 172.21.15.99:0/479822089 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f0084037020 con 0x560c9406fad0 2024-05-14T17:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 10 client.16410 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 20 client.16410 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c940ea730 0x560c940ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 4 client.16410 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c93fd3360 con 0x560c940ea730 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.004+0000 7f00a0952640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c940ea730 0x560c940ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.005+0000 7f00a0952640 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c940ea730 0x560c940ecb50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5543ebdb6754de02 server_cookie=4406b6c9197afd7a in_seq=0 out_seq=0 2024-05-14T17:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.005+0000 7f007bfff640 10 client.16410 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f007bfff640 1 -- 172.21.15.99:0/479822089 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560c93fd3360 con 0x560c940ea730 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f007bfff640 10 client.16410 handle_command_reply: tid=0 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 client.16410 shutdown 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 msgr2=0x560c94073760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c94073760 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0084004de0 tx=0x7f0084012640 comp rx=0 tx=0).stop 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 shutdown_connections 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x560c940ea730 0x560c940ecb50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f007bfff640 0 client.16410 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560c9406fad0 0x560c94073760 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c94065210 0x560c94073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 --2- 172.21.15.99:0/479822089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560c940648f0 0x560c94077e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 >> 172.21.15.99:0/479822089 conn(0x560c94060e00 msgr2=0x560c940673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.059+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 shutdown_connections 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.060+0000 7f00a59a9740 1 -- 172.21.15.99:0/479822089 wait complete. 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.060+0000 7f00a59a9740 20 client.16410 trim_cache size 0 max 16384 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:13:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.060+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00880b2a90 msgr2=0x7f00880b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.060+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00880b2a90 0x7f00880b4f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0084009fd0 tx=0x7f0084005f90 comp rx=0 tx=0).stop 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.060+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 msgr2=0x7f009c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.060+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c198040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0094007b50 tx=0x7f009400e420 comp rx=0 tx=0).stop 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 shutdown_connections 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00880b2a90 0x7f00880b4f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f009c107940 0x7f009c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f009c102340 0x7f009c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 --2- 172.21.15.99:0/672701878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f009c101980 0x7f009c198040 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 >> 172.21.15.99:0/672701878 conn(0x7f009c078ad0 msgr2=0x7f009c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.061+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 shutdown_connections 2024-05-14T17:13:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:03.062+0000 7f00a2bb6640 1 -- 172.21.15.99:0/672701878 wait complete. 2024-05-14T17:13:03.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:02 smithi099 ceph-mon[31851]: pgmap v293: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-05-14T17:13:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:02 smithi193 ceph-mon[35962]: pgmap v293: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-05-14T17:13:03.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:02 smithi150 ceph-mon[36204]: pgmap v293: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-05-14T17:13:03.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 414393 4.71 MB/sec execute 315 sec latency 1460.272 ms 2024-05-14T17:13:04.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 416300 4.73 MB/sec execute 316 sec latency 23.127 ms 2024-05-14T17:13:05.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:04 smithi099 ceph-mon[31851]: pgmap v294: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-05-14T17:13:05.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:04 smithi193 ceph-mon[35962]: pgmap v294: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-05-14T17:13:05.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:04 smithi150 ceph-mon[36204]: pgmap v294: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-05-14T17:13:05.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 418313 4.75 MB/sec execute 317 sec latency 155.992 ms 2024-05-14T17:13:06.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 419527 4.75 MB/sec execute 318 sec latency 20.564 ms 2024-05-14T17:13:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:06 smithi099 ceph-mon[31851]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-14T17:13:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:06 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:06 smithi099 ceph-mon[31851]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:13:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:06 smithi193 ceph-mon[35962]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-14T17:13:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:06 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:06 smithi193 ceph-mon[35962]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:13:07.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:06 smithi150 ceph-mon[36204]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-05-14T17:13:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:06 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:06 smithi150 ceph-mon[36204]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:13:07.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 4.75 MB/sec execute 319 sec latency 651.484 ms 2024-05-14T17:13:08.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 4.74 MB/sec execute 320 sec latency 1651.538 ms 2024-05-14T17:13:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:08 smithi099 ceph-mon[31851]: pgmap v297: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-05-14T17:13:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:08 smithi193 ceph-mon[35962]: pgmap v297: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-05-14T17:13:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:08 smithi150 ceph-mon[36204]: pgmap v297: 97 pgs: 97 active+clean; 106 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-05-14T17:13:09.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 420275 4.72 MB/sec execute 321 sec latency 2651.657 ms 2024-05-14T17:13:10.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 422530 4.75 MB/sec execute 322 sec latency 2706.119 ms 2024-05-14T17:13:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:10 smithi099 ceph-mon[31851]: pgmap v298: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-14T17:13:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:10 smithi193 ceph-mon[35962]: pgmap v298: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-14T17:13:11.311 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-05-14T17:13:11.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:13:11.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:10 smithi150 ceph-mon[36204]: pgmap v298: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-14T17:13:11.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423824 4.75 MB/sec execute 323 sec latency 303.786 ms 2024-05-14T17:13:11.564 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:13:12.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:11 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:12.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:11 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:12.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:11 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:12.545 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423824 4.74 MB/sec execute 324 sec latency 1303.830 ms 2024-05-14T17:13:13.103 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:12 smithi099 ceph-mon[31851]: pgmap v299: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-14T17:13:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.205+0000 7f2ec81c2700 1 -- 172.21.15.99:0/3062903514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 msgr2=0x7f2ec0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.205+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/3062903514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec0103bd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0099c0 tx=0x7f2ebc009cd0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 -- 172.21.15.99:0/3062903514 shutdown_connections 2024-05-14T17:13:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/3062903514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec01088f0 0x7f2ec0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/3062903514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec0103bd0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/3062903514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec01041a0 0x7f2ec01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 -- 172.21.15.99:0/3062903514 >> 172.21.15.99:0/3062903514 conn(0x7f2ec007a6a0 msgr2=0x7f2ec007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 -- 172.21.15.99:0/3062903514 shutdown_connections 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 -- 172.21.15.99:0/3062903514 wait complete. 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 Processor -- start 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 -- start start 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec01041a0 0x7f2ec019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.206+0000 7f2ec81c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec01088f0 0x7f2ec01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec81c2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01136e0 con 0x7f2ec01037f0 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec81c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0113560 con 0x7f2ec01041a0 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec81c2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0113860 con 0x7f2ec01088f0 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37462/0 (socket says 172.21.15.99:37462) 2024-05-14T17:13:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 -- 172.21.15.99:0/604416791 learned_addr learned my addr 172.21.15.99:0/604416791 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec575d700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec01041a0 0x7f2ec019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec675f700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec01088f0 0x7f2ec01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 -- 172.21.15.99:0/604416791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec01088f0 msgr2=0x7f2ec01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec01088f0 0x7f2ec01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 -- 172.21.15.99:0/604416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec01041a0 msgr2=0x7f2ec019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec01041a0 0x7f2ec019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 -- 172.21.15.99:0/604416791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec019a100 con 0x7f2ec01037f0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec5f5e700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec019f7e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00b580 tx=0x7f2ebc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc027070 con 0x7f2ec01037f0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec019a390 con 0x7f2ec01037f0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ebc004020 con 0x7f2ec01037f0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc022e30 con 0x7f2ec01037f0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.207+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01a66b0 con 0x7f2ec01037f0 2024-05-14T17:13:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.208+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec0104c80 con 0x7f2ec01037f0 2024-05-14T17:13:13.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.209+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2ebc022590 con 0x7f2ec01037f0 2024-05-14T17:13:13.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.210+0000 7f2eb6ffd700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eac0b29f0 0x7f2eac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:13.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.210+0000 7f2ec575d700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eac0b29f0 0x7f2eac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:13.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.210+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f2ebc0e3300 con 0x7f2ec01037f0 2024-05-14T17:13:13.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.211+0000 7f2ec575d700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eac0b29f0 0x7f2eac0b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0005950 tx=0x7f2eb00058e0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:13.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.215+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ebc013070 con 0x7f2ec01037f0 2024-05-14T17:13:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:12 smithi193 ceph-mon[35962]: pgmap v299: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-14T17:13:13.354 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:12 smithi150 ceph-mon[36204]: pgmap v299: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-14T17:13:13.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.469+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec002e560 con 0x7f2ec01037f0 2024-05-14T17:13:13.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.470+0000 7f2eb6ffd700 1 -- 172.21.15.99:0/604416791 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11879 (secure 0 0 0) 0x7f2ebc0aad40 con 0x7f2ec01037f0 2024-05-14T17:13:13.471 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:13:13.471 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:13:10.003659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:13:13.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.472+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eac0b29f0 msgr2=0x7f2eac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:13.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.472+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eac0b29f0 0x7f2eac0b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0005950 tx=0x7f2eb00058e0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.472+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 msgr2=0x7f2ec019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.472+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec019f7e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00b580 tx=0x7f2ebc0047b0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.472+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 shutdown_connections 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f2eac0b29f0 0x7f2eac0b4eb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ec01088f0 0x7f2ec01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec01037f0 0x7f2ec019f7e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 --2- 172.21.15.99:0/604416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec01041a0 0x7f2ec019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 >> 172.21.15.99:0/604416791 conn(0x7f2ec007a6a0 msgr2=0x7f2ec0109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 shutdown_connections 2024-05-14T17:13:13.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:13.473+0000 7f2ec81c2700 1 -- 172.21.15.99:0/604416791 wait complete. 2024-05-14T17:13:13.475 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 38 2024-05-14T17:13:13.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423824 4.72 MB/sec execute 325 sec latency 2303.900 ms 2024-05-14T17:13:14.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:13 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/604416791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:13:14.102 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-05-14T17:13:14.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-05-14T17:13:14.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:13 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/604416791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:13:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:13:14.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:13 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/604416791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:13:14.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 423824 4.71 MB/sec execute 326 sec latency 3303.927 ms 2024-05-14T17:13:15.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:14 smithi099 ceph-mon[31851]: pgmap v300: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-05-14T17:13:15.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:14 smithi099 ceph-mon[31851]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:13:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:14 smithi193 ceph-mon[35962]: pgmap v300: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-05-14T17:13:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:14 smithi193 ceph-mon[35962]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:13:15.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:14 smithi150 ceph-mon[36204]: pgmap v300: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-05-14T17:13:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:14 smithi150 ceph-mon[36204]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:13:15.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426122 4.74 MB/sec execute 327 sec latency 3326.467 ms 2024-05-14T17:13:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 -- 172.21.15.99:0/1438174962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 msgr2=0x7f5ab810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/1438174962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab810f310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800b0a0 tx=0x7f5aa800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 -- 172.21.15.99:0/1438174962 shutdown_connections 2024-05-14T17:13:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/1438174962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab810f310 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/1438174962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 0x7f5ab81088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/1438174962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 0x7f5ab81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 -- 172.21.15.99:0/1438174962 >> 172.21.15.99:0/1438174962 conn(0x7f5ab8079460 msgr2=0x7f5ab8079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 -- 172.21.15.99:0/1438174962 shutdown_connections 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.980+0000 7f5abd5d5700 1 -- 172.21.15.99:0/1438174962 wait complete. 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 Processor -- start 2024-05-14T17:13:15.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 -- start start 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 0x7f5ab819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 0x7f5ab81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8112650 con 0x7f5ab8102e20 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81124d0 con 0x7f5ab8102460 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5abd5d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81127d0 con 0x7f5ab81084d0 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50740/0 (socket says 172.21.15.99:50740) 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 0x7f5ab81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.981+0000 7f5ab67fc700 1 -- 172.21.15.99:0/2593596 learned_addr learned my addr 172.21.15.99:0/2593596 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 0x7f5ab819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab67fc700 1 -- 172.21.15.99:0/2593596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 msgr2=0x7f5ab81a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 0x7f5ab81a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab67fc700 1 -- 172.21.15.99:0/2593596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 msgr2=0x7f5ab819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 0x7f5ab819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab67fc700 1 -- 172.21.15.99:0/2593596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab81a0e00 con 0x7f5ab8102e20 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 0x7f5ab819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab77fe700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 0x7f5ab81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab819c330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000d980 tx=0x7f5aa000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5aaffff700 1 -- 172.21.15.99:0/2593596 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0016070 con 0x7f5ab8102e20 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab81a10f0 con 0x7f5ab8102e20 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5aaffff700 1 -- 172.21.15.99:0/2593596 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa0004830 con 0x7f5ab8102e20 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5aaffff700 1 -- 172.21.15.99:0/2593596 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00054c0 con 0x7f5ab8102e20 2024-05-14T17:13:15.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.982+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81a8970 con 0x7f5ab8102e20 2024-05-14T17:13:15.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.985+0000 7f5aaffff700 1 -- 172.21.15.99:0/2593596 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5aa0004220 con 0x7f5ab8102e20 2024-05-14T17:13:15.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.985+0000 7f5aaffff700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f5aa40b2aa0 0x7f5aa40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.986+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f5aa40b2aa0 0x7f5aa40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.986+0000 7f5aaffff700 1 -- 172.21.15.99:0/2593596 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f5aa00e24b0 con 0x7f5ab8102e20 2024-05-14T17:13:15.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.986+0000 7f5ac7e6bb80 1 Processor -- start 2024-05-14T17:13:15.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.986+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f5aa40b2aa0 0x7f5aa40b4f60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009c90 tx=0x7f5ab0009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:15.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.986+0000 7f5ac7e6bb80 1 -- start start 2024-05-14T17:13:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd738890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd733d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd7347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561dbd691f00 con 0x561dbd734350 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561dbd681aa0 con 0x561dbd738dd0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561dbd685ae0 con 0x561dbd7384b0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd7347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd738890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd7347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50750/0 (socket says 172.21.15.99:50750) 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab77fe700 1 -- 172.21.15.99:0/2609969105 learned_addr learned my addr 172.21.15.99:0/2609969105 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd733d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2609969105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 msgr2=0x561dbd733d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd733d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2609969105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 msgr2=0x561dbd7347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd7347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2609969105 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbd62b140 con 0x561dbd7384b0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd738890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0010a40 tx=0x7f5ab0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2026ab4e14d712d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:15.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.987+0000 7f5aad7fa700 1 -- 172.21.15.99:0/2609969105 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0011bb0 con 0x561dbd7384b0 2024-05-14T17:13:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5aad7fa700 1 -- 172.21.15.99:0/2609969105 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab0011d50 con 0x561dbd7384b0 2024-05-14T17:13:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5aad7fa700 1 -- 172.21.15.99:0/2609969105 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab001c610 con 0x561dbd7384b0 2024-05-14T17:13:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2609969105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 msgr2=0x561dbd738890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:15.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd738890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0010a40 tx=0x7f5ab0013000 comp rx=0 tx=0).stop 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2609969105 shutdown_connections 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd7347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd738890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2609969105 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd733d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2609969105 >> 172.21.15.99:0/2609969105 conn(0x561dbd660f40 msgr2=0x561dbd72e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2609969105 shutdown_connections 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.988+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2609969105 wait complete. 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 -- start start 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 Processor -- start 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd73fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd73cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561dbd588130 con 0x561dbd7384b0 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561dbd5e9f30 con 0x561dbd738dd0 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561dbd740910 con 0x561dbd734350 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd73cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ac7e6bb80 10 client.0 _ll_register_callbacks cb 0x561dbd61cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd73fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd73cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50752/0 (socket says 172.21.15.99:50752) 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ab67fc700 1 -- 172.21.15.99:0/3034227661 learned_addr learned my addr 172.21.15.99:0/3034227661 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.989+0000 7f5ab77fe700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/3034227661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 msgr2=0x561dbd73d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/3034227661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 msgr2=0x561dbd73cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd73cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/3034227661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbd5d1a60 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd73cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd73fc40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009b50 tx=0x7f5ab0012f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afb49f1a37bfc59 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5a9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3034227661 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0035060 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab0011d50 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3034227661 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f5ab0009dd0 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3034227661 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5ab0004920 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ac7e6bb80 10 client.25837 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561dbd63b8f0 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.990+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3034227661 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab0011d50 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3034227661 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f5ab001eb50 con 0x561dbd734350 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ac7e6bb80 10 client.25837 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ac7e6bb80 10 client.25837 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:13:15.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ac7e6bb80 20 client.25837 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd74f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ac7e6bb80 4 client.25837 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dbd693e50 con 0x561dbd74d470 2024-05-14T17:13:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd74f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd74f860 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6708605c11bca086 server_cookie=6b04fc9dfcb795b1 in_seq=0 out_seq=0 2024-05-14T17:13:15.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.991+0000 7f5a9e7fc700 10 client.25837 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:15.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3034227661 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5aa0083e50 con 0x561dbd74d470 2024-05-14T17:13:15.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5a9e7fc700 10 client.25837 handle_command_reply: tid=0 2024-05-14T17:13:15.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 client.25837 shutdown 2024-05-14T17:13:15.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 msgr2=0x561dbd73fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:15.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd73fc40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009b50 tx=0x7f5ab0012f10 comp rx=0 tx=0).stop 2024-05-14T17:13:15.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 shutdown_connections 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd73fc40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd74f860 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5a9e7fc700 0 client.25837 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3034227661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd73cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 >> 172.21.15.99:0/3034227661 conn(0x561dbd660f40 msgr2=0x561dbd729050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.993+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 shutdown_connections 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.994+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3034227661 wait complete. 2024-05-14T17:13:15.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:15.994+0000 7f5ac7e6bb80 20 client.25837 trim_cache size 0 max 16384 2024-05-14T17:13:16.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.013+0000 7f5ac7e6bb80 1 Processor -- start 2024-05-14T17:13:16.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 -- start start 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd733390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd729160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81b29e0 con 0x561dbd734350 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81b01b0 con 0x561dbd738dd0 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81ae020 con 0x561dbd7384b0 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd733390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd733390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50758/0 (socket says 172.21.15.99:50758) 2024-05-14T17:13:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd729160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2295460786 learned_addr learned my addr 172.21.15.99:0/2295460786 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab77fe700 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2295460786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 msgr2=0x561dbd729160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd729160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2295460786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 msgr2=0x561dbd73a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/2295460786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbd73df20 con 0x561dbd734350 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd729160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.014+0000 7f5ab77fe700 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.015+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd733390 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5ab001ef70 tx=0x7f5ab0009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f48a261b651eff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.015+0000 7f5aacff9700 1 -- 172.21.15.99:0/2295460786 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0035070 con 0x561dbd734350 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.015+0000 7f5aacff9700 1 -- 172.21.15.99:0/2295460786 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab0038bf0 con 0x561dbd734350 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.015+0000 7f5aacff9700 1 -- 172.21.15.99:0/2295460786 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0038610 con 0x561dbd734350 2024-05-14T17:13:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2295460786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 msgr2=0x561dbd733390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd733390 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5ab001ef70 tx=0x7f5ab0009d70 comp rx=0 tx=0).stop 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2295460786 shutdown_connections 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd73a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd734350 0x561dbd733390 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/2295460786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd7384b0 0x561dbd729160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2295460786 >> 172.21.15.99:0/2295460786 conn(0x561dbd660f40 msgr2=0x561dbd737a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2295460786 shutdown_connections 2024-05-14T17:13:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/2295460786 wait complete. 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- start start 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 Processor -- start 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd752f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd751310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd751a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81ae670 con 0x561dbd7384b0 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81afae0 con 0x561dbd738dd0 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81aed40 con 0x561dbd734350 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.016+0000 7f5ac7e6bb80 10 client.0 _ll_register_callbacks cb 0x561dbd61cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd751310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd752f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd751310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50770/0 (socket says 172.21.15.99:50770) 2024-05-14T17:13:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab67fc700 1 -- 172.21.15.99:0/3226287011 learned_addr learned my addr 172.21.15.99:0/3226287011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab77fe700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd751a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/3226287011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 msgr2=0x561dbd751a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd751a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/3226287011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 msgr2=0x561dbd751310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd751310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 -- 172.21.15.99:0/3226287011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbd7b79a0 con 0x561dbd734350 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab77fe700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd751a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd751310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ab6ffd700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd752f60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0033040 tx=0x7f5ab0009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d381e4eb9bb41804 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5a9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3226287011 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0035070 con 0x561dbd734350 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3226287011 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f5ab0011d40 con 0x561dbd734350 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab0038bf0 con 0x561dbd734350 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3226287011 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5ab00383b0 con 0x561dbd734350 2024-05-14T17:13:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ac7e6bb80 10 client.25843 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.017+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561dbd7b8bb0 con 0x561dbd734350 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3226287011 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab0038bf0 con 0x561dbd734350 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3226287011 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f5ab001ea40 con 0x561dbd734350 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ac7e6bb80 10 client.25843 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ac7e6bb80 10 client.25843 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ac7e6bb80 20 client.25843 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd7bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ac7e6bb80 4 client.25843 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x561dbd68d630 con 0x561dbd74d470 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd7bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5ab67fc700 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd7bb240 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec53b93e28020a78 server_cookie=a88ec925ca370f6d in_seq=0 out_seq=0 2024-05-14T17:13:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.018+0000 7f5a9e7fc700 10 client.25843 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:16.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.099+0000 7f5a9e7fc700 1 -- 172.21.15.99:0/3226287011 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561dbd68d630 con 0x561dbd74d470 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.099+0000 7f5a9e7fc700 10 client.25843 handle_command_reply: tid=0 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.099+0000 7f5ac7e6bb80 1 client.25843 shutdown 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.099+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 msgr2=0x561dbd752f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.099+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd752f60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0033040 tx=0x7f5ab0009fd0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 shutdown_connections 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561dbd738dd0 0x561dbd751a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbd734350 0x561dbd752f60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561dbd74d470 0x561dbd7bb240 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 --2- 172.21.15.99:0/3226287011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561dbd7384b0 0x561dbd751310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5a9e7fc700 0 client.25843 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 >> 172.21.15.99:0/3226287011 conn(0x561dbd660f40 msgr2=0x561dbd73e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 shutdown_connections 2024-05-14T17:13:16.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 1 -- 172.21.15.99:0/3226287011 wait complete. 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.100+0000 7f5ac7e6bb80 20 client.25843 trim_cache size 0 max 16384 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 38, 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 30, 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "84368c3": { 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 30, 2024-05-14T17:13:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 140.4, 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f5aa40b2aa0 msgr2=0x7f5aa40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f5aa40b2aa0 0x7f5aa40b4f60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009c90 tx=0x7f5ab0009260 comp rx=0 tx=0).stop 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 msgr2=0x7f5ab819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab819c330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000d980 tx=0x7f5aa000dc90 comp rx=0 tx=0).stop 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 shutdown_connections 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f5aa40b2aa0 0x7f5aa40b4f60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ab81084d0 0x7f5ab81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ab8102460 0x7f5ab819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 --2- 172.21.15.99:0/2593596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab8102e20 0x7f5ab819c330 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 >> 172.21.15.99:0/2593596 conn(0x7f5ab8079460 msgr2=0x7f5ab806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 shutdown_connections 2024-05-14T17:13:16.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:16.101+0000 7f5abd5d5700 1 -- 172.21.15.99:0/2593596 wait complete. 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.161+0000 7ff64c746640 1 -- 172.21.15.99:0/1124773304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 msgr2=0x7ff644102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.161+0000 7ff64c746640 1 --2- 172.21.15.99:0/1124773304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644102eb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff634009930 tx=0x7ff63402a420 comp rx=0 tx=0).stop 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.161+0000 7ff64c746640 1 -- 172.21.15.99:0/1124773304 shutdown_connections 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.161+0000 7ff64c746640 1 --2- 172.21.15.99:0/1124773304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6441033f0 0x7ff64410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.161+0000 7ff64c746640 1 --2- 172.21.15.99:0/1124773304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644102eb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.161+0000 7ff64c746640 1 --2- 172.21.15.99:0/1124773304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff644108a40 0x7ff644108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- 172.21.15.99:0/1124773304 >> 172.21.15.99:0/1124773304 conn(0x7ff6440fe500 msgr2=0x7ff644100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- 172.21.15.99:0/1124773304 shutdown_connections 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- 172.21.15.99:0/1124773304 wait complete. 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 Processor -- start 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- start start 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6441033f0 0x7ff644199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff644108a40 0x7ff64419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff644112d00 con 0x7ff644102a30 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff644112b80 con 0x7ff644108a40 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64c746640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff644112e80 con 0x7ff6441033f0 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64a4e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64a4e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50790/0 (socket says 172.21.15.99:50790) 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.162+0000 7ff64a4e2640 1 -- 172.21.15.99:0/926021209 learned_addr learned my addr 172.21.15.99:0/926021209 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff649ce1640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6441033f0 0x7ff644199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64ace3640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff644108a40 0x7ff64419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64a4e2640 1 -- 172.21.15.99:0/926021209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6441033f0 msgr2=0x7ff644199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6441033f0 0x7ff644199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64a4e2640 1 -- 172.21.15.99:0/926021209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff644108a40 msgr2=0x7ff64419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff644108a40 0x7ff64419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64a4e2640 1 -- 172.21.15.99:0/926021209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64419e1f0 con 0x7ff644102a30 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644199200 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff64000d950 tx=0x7ff64000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff63b7fe640 1 -- 172.21.15.99:0/926021209 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640017030 con 0x7ff644102a30 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff63b7fe640 1 -- 172.21.15.99:0/926021209 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff640004020 con 0x7ff644102a30 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64419e4e0 con 0x7ff644102a30 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff63b7fe640 1 -- 172.21.15.99:0/926021209 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6400047a0 con 0x7ff644102a30 2024-05-14T17:13:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.163+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6441a5d70 con 0x7ff644102a30 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff63b7fe640 1 -- 172.21.15.99:0/926021209 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff640009a30 con 0x7ff644102a30 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 Processor -- start 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 -- start start 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e138f0 0x55c430e13cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 0x55c430e20ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c430d86c30 con 0x55c430e1ead0 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c430d767d0 con 0x55c430e14210 2024-05-14T17:13:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64f466740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c430d7a810 con 0x55c430e138f0 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 0x55c430e20ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 0x55c430e20ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50792/0 (socket says 172.21.15.99:50792) 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 -- 172.21.15.99:0/1574041688 learned_addr learned my addr 172.21.15.99:0/1574041688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff64ace3640 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e138f0 0x55c430e13cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 -- 172.21.15.99:0/1574041688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e138f0 msgr2=0x55c430e13cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e138f0 0x55c430e13cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 -- 172.21.15.99:0/1574041688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 msgr2=0x55c430e1e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff649ce1640 1 -- 172.21.15.99:0/1574041688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c430a8b210 con 0x55c430e1ead0 2024-05-14T17:13:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.166+0000 7ff63b7fe640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff6300b2a90 0x7ff6300b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.167+0000 7ff64ace3640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff6300b2a90 0x7ff6300b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.167+0000 7ff649ce1640 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 0x55c430e20ec0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff63402a940 tx=0x7ff634031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a014532c2c72401c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.167+0000 7ff63b7fe640 1 -- 172.21.15.99:0/926021209 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff6400e1f40 con 0x7ff644102a30 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.167+0000 7ff638ff9640 1 -- 172.21.15.99:0/1574041688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634040070 con 0x55c430e1ead0 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.167+0000 7ff638ff9640 1 -- 172.21.15.99:0/1574041688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340316a0 con 0x55c430e1ead0 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.167+0000 7ff638ff9640 1 -- 172.21.15.99:0/1574041688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634035040 con 0x55c430e1ead0 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- 172.21.15.99:0/1574041688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 msgr2=0x55c430e20ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64ace3640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff6300b2a90 0x7ff6300b4f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c009500 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 0x55c430e20ec0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff63402a940 tx=0x7ff634031bc0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- 172.21.15.99:0/1574041688 shutdown_connections 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e1ead0 0x55c430e20ec0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- 172.21.15.99:0/1574041688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e138f0 0x55c430e13cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- 172.21.15.99:0/1574041688 >> 172.21.15.99:0/1574041688 conn(0x55c430e0fe00 msgr2=0x55c430e101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- 172.21.15.99:0/1574041688 shutdown_connections 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- 172.21.15.99:0/1574041688 wait complete. 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- start start 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 Processor -- start 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e1e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c430a5c790 con 0x55c430e138f0 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c430d32890 con 0x55c430e14210 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c430e21740 con 0x55c430e1ead0 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.168+0000 7ff64f466740 10 client.0 _ll_register_callbacks cb 0x55c430d35d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e1e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e1e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50802/0 (socket says 172.21.15.99:50802) 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 -- 172.21.15.99:0/640902159 learned_addr learned my addr 172.21.15.99:0/640902159 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64ace3640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff649ce1640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 -- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 msgr2=0x55c430e1bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 -- 172.21.15.99:0/640902159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 msgr2=0x55c430e1b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 -- 172.21.15.99:0/640902159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c430b57620 con 0x55c430e138f0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff649ce1640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e1e400 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff6400a6490 tx=0x7ff64000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0950896ae95942 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff62dffb640 1 -- 172.21.15.99:0/640902159 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6400a8b70 con 0x55c430e138f0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff62dffb640 1 -- 172.21.15.99:0/640902159 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff6400df910 con 0x55c430e138f0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6340316a0 con 0x55c430e138f0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff62dffb640 1 -- 172.21.15.99:0/640902159 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff6400a8990 con 0x55c430e138f0 2024-05-14T17:13:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64f466740 10 client.16446 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c430e2d140 con 0x55c430e138f0 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.169+0000 7ff62dffb640 1 -- 172.21.15.99:0/640902159 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6340316a0 con 0x55c430e138f0 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff62dffb640 1 -- 172.21.15.99:0/640902159 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7ff6400a54b0 con 0x55c430e138f0 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff64f466740 10 client.16446 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff64f466740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff64f466740 20 client.16446 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff64f466740 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e2e650 0x55c430e30a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff64f466740 4 client.16446 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c430d88b80 con 0x55c430e2e650 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff649ce1640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e2e650 0x55c430e30a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff62dffb640 10 client.16446 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.170+0000 7ff649ce1640 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e2e650 0x55c430e30a80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c963b7483da031e0 server_cookie=ac37dcc95e23ac97 in_seq=0 out_seq=0 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff62dffb640 1 -- 172.21.15.99:0/640902159 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c430d88b80 con 0x55c430e2e650 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff62dffb640 10 client.16446 handle_command_reply: tid=0 2024-05-14T17:13:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 client.16446 shutdown 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 msgr2=0x55c430e1e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e1e400 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff6400a6490 tx=0x7ff64000b530 comp rx=0 tx=0).stop 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 shutdown_connections 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e2e650 0x55c430e30a80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e1b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff62dffb640 0 client.16446 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 --2- 172.21.15.99:0/640902159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e1e400 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 >> 172.21.15.99:0/640902159 conn(0x55c430e0fe00 msgr2=0x55c430e0f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 shutdown_connections 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 1 -- 172.21.15.99:0/640902159 wait complete. 2024-05-14T17:13:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.171+0000 7ff64f466740 20 client.16446 trim_cache size 0 max 16384 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 Processor -- start 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- start start 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e138f0 0x55c430e16dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 0x55c430e159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441b1020 con 0x55c430e14210 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441ae830 con 0x55c430e138f0 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441ac6a0 con 0x55c430e1ead0 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 0x55c430e159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 0x55c430e159b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50806/0 (socket says 172.21.15.99:50806) 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 -- 172.21.15.99:0/1085169461 learned_addr learned my addr 172.21.15.99:0/1085169461 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64ace3640 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e138f0 0x55c430e16dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 -- 172.21.15.99:0/1085169461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 msgr2=0x55c430e1df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 -- 172.21.15.99:0/1085169461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e138f0 msgr2=0x55c430e16dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e138f0 0x55c430e16dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 -- 172.21.15.99:0/1085169461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c430e0fc30 con 0x55c430e14210 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff649ce1640 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 0x55c430e159b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff634009930 tx=0x7ff634005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e55f253ffe17d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff62ffff640 1 -- 172.21.15.99:0/1085169461 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634035040 con 0x55c430e14210 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff62ffff640 1 -- 172.21.15.99:0/1085169461 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff634031860 con 0x55c430e14210 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff62ffff640 1 -- 172.21.15.99:0/1085169461 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634040040 con 0x55c430e14210 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- 172.21.15.99:0/1085169461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 msgr2=0x55c430e159b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 0x55c430e159b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff634009930 tx=0x7ff634005020 comp rx=0 tx=0).stop 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- 172.21.15.99:0/1085169461 shutdown_connections 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e1df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e14210 0x55c430e159b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 --2- 172.21.15.99:0/1085169461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e138f0 0x55c430e16dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- 172.21.15.99:0/1085169461 >> 172.21.15.99:0/1085169461 conn(0x55c430e0fe00 msgr2=0x55c430e152a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- 172.21.15.99:0/1085169461 shutdown_connections 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.188+0000 7ff64f466740 1 -- 172.21.15.99:0/1085169461 wait complete. 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 -- start start 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 Processor -- start 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e22a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441accf0 con 0x55c430e138f0 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441ae160 con 0x55c430e14210 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441ad3c0 con 0x55c430e1ead0 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64f466740 10 client.0 _ll_register_callbacks cb 0x7ff6400044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e22530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50808/0 (socket says 172.21.15.99:50808) 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 -- 172.21.15.99:0/3514659867 learned_addr learned my addr 172.21.15.99:0/3514659867 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64ace3640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 -- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 msgr2=0x55c430e22fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e22fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 -- 172.21.15.99:0/3514659867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 msgr2=0x55c430e22a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e22a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 -- 172.21.15.99:0/3514659867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c430e97f40 con 0x55c430e138f0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64ace3640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff64a4e2640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e22530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff6400dfc90 tx=0x7ff6400a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505ddca9430dde88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.189+0000 7ff62dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff62dffb640 1 -- 172.21.15.99:0/3514659867 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6400a8b70 con 0x55c430e138f0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff62dffb640 1 -- 172.21.15.99:0/3514659867 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff6400e5dd0 con 0x55c430e138f0 2024-05-14T17:13:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff634031860 con 0x55c430e138f0 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff62dffb640 1 -- 172.21.15.99:0/3514659867 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff6400a8d30 con 0x55c430e138f0 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 10 client.16452 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c430e99230 con 0x55c430e138f0 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff62dffb640 1 -- 172.21.15.99:0/3514659867 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff634031860 con 0x55c430e138f0 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff62dffb640 1 -- 172.21.15.99:0/3514659867 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7ff6400ad110 con 0x55c430e138f0 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 10 client.16452 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 20 client.16452 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e99a10 0x55c430e9be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 4 client.16452 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c430e1bfc0 con 0x55c430e99a10 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.190+0000 7ff649ce1640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e99a10 0x55c430e9be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.191+0000 7ff62dffb640 10 client.16452 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.191+0000 7ff649ce1640 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e99a10 0x55c430e9be90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a4b8f28a454e5e server_cookie=2cbbc23802a97d5f in_seq=0 out_seq=0 2024-05-14T17:13:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff62dffb640 1 -- 172.21.15.99:0/3514659867 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c430e1bfc0 con 0x55c430e99a10 2024-05-14T17:13:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff62dffb640 10 client.16452 handle_command_reply: tid=0 2024-05-14T17:13:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 client.16452 shutdown 2024-05-14T17:13:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 msgr2=0x55c430e22530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e22530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff6400dfc90 tx=0x7ff6400a56c0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 shutdown_connections 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55c430e99a10 0x55c430e9be90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c430e1ead0 0x55c430e22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c430e14210 0x55c430e22a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 --2- 172.21.15.99:0/3514659867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c430e138f0 0x55c430e22530 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff62dffb640 0 client.16452 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 >> 172.21.15.99:0/3514659867 conn(0x55c430e0fe00 msgr2=0x55c430e15b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 shutdown_connections 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 1 -- 172.21.15.99:0/3514659867 wait complete. 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64f466740 20 client.16452 trim_cache size 0 max 16384 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:13:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:13:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff6300b2a90 msgr2=0x7ff6300b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff6300b2a90 0x7ff6300b4f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c009500 comp rx=0 tx=0).stop 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 msgr2=0x7ff644199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644199200 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff64000d950 tx=0x7ff64000de10 comp rx=0 tx=0).stop 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 shutdown_connections 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff6300b2a90 0x7ff6300b4f50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff644108a40 0x7ff64419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6441033f0 0x7ff644199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 --2- 172.21.15.99:0/926021209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff644102a30 0x7ff644199200 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 >> 172.21.15.99:0/926021209 conn(0x7ff6440fe500 msgr2=0x7ff6440fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 shutdown_connections 2024-05-14T17:13:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:16.239+0000 7ff64c746640 1 -- 172.21.15.99:0/926021209 wait complete. 2024-05-14T17:13:16.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.73 MB/sec execute 328 sec latency 447.843 ms 2024-05-14T17:13:16.685 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '84368c3', quiesce took 5.4 seconds. Will release after: 14.999999999999998 2024-05-14T17:13:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:17 smithi099 ceph-mon[31851]: pgmap v302: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-14T17:13:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:17 smithi193 ceph-mon[35962]: pgmap v302: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-14T17:13:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:17 smithi150 ceph-mon[36204]: pgmap v302: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-14T17:13:17.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.71 MB/sec execute 329 sec latency 1447.947 ms 2024-05-14T17:13:18.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.70 MB/sec execute 330 sec latency 2447.991 ms 2024-05-14T17:13:19.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:19 smithi099 ceph-mon[31851]: pgmap v303: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-05-14T17:13:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:19 smithi193 ceph-mon[35962]: pgmap v303: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-05-14T17:13:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:19 smithi150 ceph-mon[36204]: pgmap v303: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-05-14T17:13:19.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.68 MB/sec execute 331 sec latency 3448.024 ms 2024-05-14T17:13:20.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.67 MB/sec execute 332 sec latency 4448.060 ms 2024-05-14T17:13:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:21 smithi099 ceph-mon[31851]: pgmap v304: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:13:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:21 smithi193 ceph-mon[35962]: pgmap v304: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:13:21.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:21 smithi150 ceph-mon[36204]: pgmap v304: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:13:21.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.66 MB/sec execute 333 sec latency 5448.137 ms 2024-05-14T17:13:22.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.64 MB/sec execute 334 sec latency 6448.248 ms 2024-05-14T17:13:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:23 smithi099 ceph-mon[31851]: pgmap v305: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:13:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:23 smithi193 ceph-mon[35962]: pgmap v305: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:13:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:23 smithi150 ceph-mon[36204]: pgmap v305: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:13:23.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.63 MB/sec execute 335 sec latency 7448.339 ms 2024-05-14T17:13:24.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.61 MB/sec execute 336 sec latency 8448.471 ms 2024-05-14T17:13:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:25 smithi099 ceph-mon[31851]: pgmap v306: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-05-14T17:13:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:25 smithi193 ceph-mon[35962]: pgmap v306: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-05-14T17:13:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:25 smithi150 ceph-mon[36204]: pgmap v306: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-05-14T17:13:25.546 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.60 MB/sec execute 337 sec latency 9448.562 ms 2024-05-14T17:13:26.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.59 MB/sec execute 338 sec latency 10448.668 ms 2024-05-14T17:13:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:27 smithi193 ceph-mon[35962]: pgmap v307: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1022 KiB/s wr, 102 op/s 2024-05-14T17:13:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:27 smithi150 ceph-mon[36204]: pgmap v307: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1022 KiB/s wr, 102 op/s 2024-05-14T17:13:27.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:27 smithi099 ceph-mon[31851]: pgmap v307: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1022 KiB/s wr, 102 op/s 2024-05-14T17:13:27.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.57 MB/sec execute 339 sec latency 11448.794 ms 2024-05-14T17:13:28.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.56 MB/sec execute 340 sec latency 12448.907 ms 2024-05-14T17:13:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:29 smithi193 ceph-mon[35962]: pgmap v308: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 923 KiB/s wr, 92 op/s 2024-05-14T17:13:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.337+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1405527234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 msgr2=0x7f0bbc10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.337+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1405527234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc10e830 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009cd0 tx=0x7f0bb402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1405527234 shutdown_connections 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1405527234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc10e830 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1405527234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1405527234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bbc107970 0x7f0bbc107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1405527234 >> 172.21.15.99:0/1405527234 conn(0x7f0bbc078b60 msgr2=0x7f0bbc078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1405527234 shutdown_connections 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1405527234 wait complete. 2024-05-14T17:13:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.338+0000 7f0bc0d51640 1 Processor -- start 2024-05-14T17:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 -- start start 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bbc107970 0x7f0bbc1a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc111c00 con 0x7f0bbc102370 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc111a80 con 0x7f0bbc107970 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bc0d51640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc111d80 con 0x7f0bbc1019b0 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60462/0 (socket says 172.21.15.99:60462) 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/1599571545 learned_addr learned my addr 172.21.15.99:0/1599571545 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bba59c640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bbc107970 0x7f0bbc1a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/1599571545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bbc1019b0 msgr2=0x7f0bbc199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/1599571545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bbc107970 msgr2=0x7f0bbc1a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bbc107970 0x7f0bbc1a4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.339+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/1599571545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc1a54a0 con 0x7f0bbc102370 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.340+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc19a420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e9f0 tx=0x7f0ba400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.340+0000 7f0bb3fff640 1 -- 172.21.15.99:0/1599571545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400cd80 con 0x7f0bbc102370 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.340+0000 7f0bb3fff640 1 -- 172.21.15.99:0/1599571545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba40108e0 con 0x7f0bbc102370 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.340+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc1a5670 con 0x7f0bbc102370 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.340+0000 7f0bb3fff640 1 -- 172.21.15.99:0/1599571545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4018980 con 0x7f0bbc102370 2024-05-14T17:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.340+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a5bc0 con 0x7f0bbc102370 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.343+0000 7f0bc3a6c740 1 Processor -- start 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.343+0000 7f0bb3fff640 1 -- 172.21.15.99:0/1599571545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0ba4010430 con 0x7f0bbc102370 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.343+0000 7f0bc3a6c740 1 -- start start 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.343+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 0x55cc1a48bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.343+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48c210 0x55cc1a496590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a496ad0 0x55cc1a498ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc1a3fec30 con 0x55cc1a48b8f0 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cc1a3ee7d0 con 0x55cc1a48c210 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cc1a3f2810 con 0x55cc1a496ad0 2024-05-14T17:13:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 0x55cc1a48bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 0x55cc1a48bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60466/0 (socket says 172.21.15.99:60466) 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 -- 172.21.15.99:0/3734548726 learned_addr learned my addr 172.21.15.99:0/3734548726 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bba59c640 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48c210 0x55cc1a496590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a496ad0 0x55cc1a498ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 -- 172.21.15.99:0/3734548726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a496ad0 msgr2=0x55cc1a498ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a496ad0 0x55cc1a498ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 -- 172.21.15.99:0/3734548726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48c210 msgr2=0x55cc1a496590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48c210 0x55cc1a496590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 -- 172.21.15.99:0/3734548726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1a103210 con 0x55cc1a48b8f0 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 0x55cc1a48bcd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009df0 tx=0x7f0bb4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10096c1375078a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bb3fff640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0ba80b2a10 0x7f0ba80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bba59c640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0ba80b2a10 0x7f0ba80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bb0d99640 1 -- 172.21.15.99:0/3734548726 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4031850 con 0x55cc1a48b8f0 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bb0d99640 1 -- 172.21.15.99:0/3734548726 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4036040 con 0x55cc1a48b8f0 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.344+0000 7f0bb0d99640 1 -- 172.21.15.99:0/3734548726 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40043c0 con 0x55cc1a48b8f0 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bb3fff640 1 -- 172.21.15.99:0/1599571545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f0ba40e2aa0 con 0x7f0bbc102370 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/3734548726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 msgr2=0x55cc1a48bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bba59c640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0ba80b2a10 0x7f0ba80b4ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55cc1a48ce90 tx=0x7f0bac00c430 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 0x55cc1a48bcd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009df0 tx=0x7f0bb4005f70 comp rx=0 tx=0).stop 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/3734548726 shutdown_connections 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a496ad0 0x55cc1a498ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48c210 0x55cc1a496590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/3734548726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48b8f0 0x55cc1a48bcd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/3734548726 >> 172.21.15.99:0/3734548726 conn(0x55cc1a487e00 msgr2=0x55cc1a4881e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/3734548726 shutdown_connections 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.345+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/3734548726 wait complete. 2024-05-14T17:13:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 -- start start 2024-05-14T17:13:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 Processor -- start 2024-05-14T17:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a496480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a493770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a496ad0 0x55cc1a493d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc1a0d4790 con 0x55cc1a48c210 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cc1a3aa890 con 0x55cc1a496ad0 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cc1a499740 con 0x55cc1a48b8f0 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bc3a6c740 10 client.0 _ll_register_callbacks cb 0x55cc1a3add60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a493770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a493770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60472/0 (socket says 172.21.15.99:60472) 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bba59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a496480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/156859952 learned_addr learned my addr 172.21.15.99:0/156859952 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a496ad0 0x55cc1a493d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 msgr2=0x55cc1a496480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a496480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/156859952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a496ad0 msgr2=0x55cc1a493d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a496ad0 0x55cc1a493d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.346+0000 7f0bb1d9b640 1 -- 172.21.15.99:0/156859952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1a1cf620 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a493770 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40a6ff0 tx=0x7f0ba40e61b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2677d48c709fa6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0ba1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/156859952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40e8030 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/156859952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f0ba40a6370 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb4036040 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/156859952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0ba40a9890 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 10 client.16464 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cc1a4a51f0 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/156859952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4036040 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/156859952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f0ba4018b20 con 0x55cc1a48c210 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 10 client.16464 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 20 client.16464 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a4a6770 0x55cc1a4a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 4 client.16464 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.347+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc1a400b80 con 0x55cc1a4a6770 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.348+0000 7f0bba59c640 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a4a6770 0x55cc1a4a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.348+0000 7f0ba1ffb640 10 client.16464 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.348+0000 7f0bba59c640 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a4a6770 0x55cc1a4a8b60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d75bc276b50c88cb server_cookie=6a31f0d0c0ac02f1 in_seq=0 out_seq=0 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/156859952 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc1a400b80 con 0x55cc1a4a6770 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0ba1ffb640 10 client.16464 handle_command_reply: tid=0 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 client.16464 shutdown 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 msgr2=0x55cc1a493770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a493770 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40a6ff0 tx=0x7f0ba40e61b0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 shutdown_connections 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a4a6770 0x55cc1a4a8b60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a496ad0 0x55cc1a493d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0ba1ffb640 0 client.16464 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a493770 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/156859952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a496480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 >> 172.21.15.99:0/156859952 conn(0x55cc1a487e00 msgr2=0x55cc1a487100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 shutdown_connections 2024-05-14T17:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/156859952 wait complete. 2024-05-14T17:13:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.349+0000 7f0bc3a6c740 20 client.16464 trim_cache size 0 max 16384 2024-05-14T17:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 Processor -- start 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 -- start start 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a487e00 0x55cc1a495fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48b8f0 0x55cc1a4ae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a4ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1b0e30 con 0x55cc1a48c210 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1ae640 con 0x55cc1a48b8f0 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1ac4b0 con 0x55cc1a487e00 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a4ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a4ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60486/0 (socket says 172.21.15.99:60486) 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.364+0000 7f0bbad9d640 1 -- 172.21.15.99:0/1749881618 learned_addr learned my addr 172.21.15.99:0/1749881618 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bba59c640 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a487e00 0x55cc1a495fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bbad9d640 1 -- 172.21.15.99:0/1749881618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a487e00 msgr2=0x55cc1a495fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a487e00 0x55cc1a495fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bbad9d640 1 -- 172.21.15.99:0/1749881618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48b8f0 msgr2=0x55cc1a4ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48b8f0 0x55cc1a4ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48b8f0 0x55cc1a4ae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bbad9d640 1 -- 172.21.15.99:0/1749881618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1a48c040 con 0x55cc1a48c210 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bba59c640 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a487e00 0x55cc1a495fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a4ae940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009df0 tx=0x7f0bb4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c960631808954bfd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0ba3fff640 1 -- 172.21.15.99:0/1749881618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4031a50 con 0x55cc1a48c210 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0ba3fff640 1 -- 172.21.15.99:0/1749881618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4036040 con 0x55cc1a48c210 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.365+0000 7f0ba3fff640 1 -- 172.21.15.99:0/1749881618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004600 con 0x55cc1a48c210 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/1749881618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 msgr2=0x55cc1a4ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a4ae940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009df0 tx=0x7f0bb4000f40 comp rx=0 tx=0).stop 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/1749881618 shutdown_connections 2024-05-14T17:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a4ae940 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a48b8f0 0x55cc1a4ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/1749881618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a487e00 0x55cc1a495fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/1749881618 >> 172.21.15.99:0/1749881618 conn(0x55cc1a483400 msgr2=0x55cc1a48db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/1749881618 shutdown_connections 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/1749881618 wait complete. 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- start start 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 Processor -- start 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a487e00 0x55cc1a49ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a49b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a50ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1acb00 con 0x55cc1a48c210 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1adf70 con 0x55cc1a487e00 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1ad1d0 con 0x55cc1a48b8f0 2024-05-14T17:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bc3a6c740 10 client.0 _ll_register_callbacks cb 0x7f0ba40a5d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a50ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bbad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a50ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60488/0 (socket says 172.21.15.99:60488) 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bbad9d640 1 -- 172.21.15.99:0/480163142 learned_addr learned my addr 172.21.15.99:0/480163142 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a49b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.366+0000 7f0bba59c640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a487e00 0x55cc1a49ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bbad9d640 1 -- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 msgr2=0x55cc1a49b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a49b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bbad9d640 1 -- 172.21.15.99:0/480163142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a487e00 msgr2=0x55cc1a49ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a487e00 0x55cc1a49ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bbad9d640 1 -- 172.21.15.99:0/480163142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1a5104f0 con 0x55cc1a48c210 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bb1d9b640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a49b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bbad9d640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a50ffb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40096a0 tx=0x7f0bb4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1279ff0d44c7f702 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0ba1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/480163142 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4003970 con 0x55cc1a48c210 2024-05-14T17:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/480163142 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f0bb4003b10 con 0x55cc1a48c210 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb4036040 con 0x55cc1a48c210 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/480163142 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0bb4003d10 con 0x55cc1a48c210 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 10 client.16470 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cc1a5117b0 con 0x55cc1a48c210 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/480163142 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb4036040 con 0x55cc1a48c210 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/480163142 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f0bb4030070 con 0x55cc1a48c210 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 10 client.16470 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 20 client.16470 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a496ad0 0x55cc1a513a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 4 client.16470 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.367+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc1a3fa360 con 0x55cc1a496ad0 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.368+0000 7f0bba59c640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a496ad0 0x55cc1a513a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.368+0000 7f0bba59c640 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a496ad0 0x55cc1a513a90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9117cc269e5b1700 server_cookie=6935be43e9e77fd2 in_seq=0 out_seq=0 2024-05-14T17:13:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.368+0000 7f0ba1ffb640 10 client.16470 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0ba1ffb640 1 -- 172.21.15.99:0/480163142 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cc1a3fa360 con 0x55cc1a496ad0 2024-05-14T17:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0ba1ffb640 10 client.16470 handle_command_reply: tid=0 2024-05-14T17:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 client.16470 shutdown 2024-05-14T17:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 msgr2=0x55cc1a50ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a50ffb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40096a0 tx=0x7f0bb4004410 comp rx=0 tx=0).stop 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 shutdown_connections 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55cc1a496ad0 0x55cc1a513a90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc1a48c210 0x55cc1a50ffb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cc1a48b8f0 0x55cc1a49b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0bc3a6c740 1 --2- 172.21.15.99:0/480163142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cc1a487e00 0x55cc1a49ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.403+0000 7f0ba1ffb640 0 client.16470 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 >> 172.21.15.99:0/480163142 conn(0x55cc1a483400 msgr2=0x55cc1a486870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 shutdown_connections 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc3a6c740 1 -- 172.21.15.99:0/480163142 wait complete. 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc3a6c740 20 client.16470 trim_cache size 0 max 16384 2024-05-14T17:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0ba80b2a10 msgr2=0x7f0ba80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0ba80b2a10 0x7f0ba80b4ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55cc1a48ce90 tx=0x7f0bac00c430 comp rx=0 tx=0).stop 2024-05-14T17:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 msgr2=0x7f0bbc19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc19a420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e9f0 tx=0x7f0ba400eeb0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 shutdown_connections 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f0ba80b2a10 0x7f0ba80b4ed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bbc107970 0x7f0bbc1a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bbc102370 0x7f0bbc19a420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 --2- 172.21.15.99:0/1599571545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 >> 172.21.15.99:0/1599571545 conn(0x7f0bbc078b60 msgr2=0x7f0bbc104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 shutdown_connections 2024-05-14T17:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:29.404+0000 7f0bc0d51640 1 -- 172.21.15.99:0/1599571545 wait complete. 2024-05-14T17:13:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:29 smithi150 ceph-mon[36204]: pgmap v308: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 923 KiB/s wr, 92 op/s 2024-05-14T17:13:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:29 smithi099 ceph-mon[31851]: pgmap v308: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 923 KiB/s wr, 92 op/s 2024-05-14T17:13:29.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.55 MB/sec execute 341 sec latency 13448.986 ms 2024-05-14T17:13:30.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.53 MB/sec execute 342 sec latency 14449.056 ms 2024-05-14T17:13:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:31 smithi193 ceph-mon[35962]: pgmap v309: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-05-14T17:13:31.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:31 smithi150 ceph-mon[36204]: pgmap v309: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-05-14T17:13:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:31 smithi099 ceph-mon[31851]: pgmap v309: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-05-14T17:13:31.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.52 MB/sec execute 343 sec latency 15449.119 ms 2024-05-14T17:13:31.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:13:31.949 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:13:32.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.51 MB/sec execute 344 sec latency 16449.200 ms 2024-05-14T17:13:33.288 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:33 smithi099 ceph-mon[31851]: pgmap v310: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 538 KiB/s wr, 29 op/s 2024-05-14T17:13:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:33 smithi193 ceph-mon[35962]: pgmap v310: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 538 KiB/s wr, 29 op/s 2024-05-14T17:13:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:33 smithi150 ceph-mon[36204]: pgmap v310: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 538 KiB/s wr, 29 op/s 2024-05-14T17:13:33.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.49 MB/sec execute 345 sec latency 17449.264 ms 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 -- 172.21.15.99:0/2434046051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 msgr2=0x7fc718108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 --2- 172.21.15.99:0/2434046051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc718108890 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009990 tx=0x7fc70c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 -- 172.21.15.99:0/2434046051 shutdown_connections 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 --2- 172.21.15.99:0/2434046051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc718102e30 0x7fc71810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 --2- 172.21.15.99:0/2434046051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc718108890 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 --2- 172.21.15.99:0/2434046051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc718102470 0x7fc7181028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 -- 172.21.15.99:0/2434046051 >> 172.21.15.99:0/2434046051 conn(0x7fc7180793f0 msgr2=0x7fc718079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 -- 172.21.15.99:0/2434046051 shutdown_connections 2024-05-14T17:13:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.616+0000 7fc71cf49700 1 -- 172.21.15.99:0/2434046051 wait complete. 2024-05-14T17:13:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 Processor -- start 2024-05-14T17:13:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 -- start start 2024-05-14T17:13:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc718102470 0x7fc71819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc718102e30 0x7fc71819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc7181a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181125e0 con 0x7fc7181084b0 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc718112460 con 0x7fc718102470 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71cf49700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc718112760 con 0x7fc718102e30 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc7181a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc7181a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60510/0 (socket says 172.21.15.99:60510) 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 -- 172.21.15.99:0/1860856062 learned_addr learned my addr 172.21.15.99:0/1860856062 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc71659c700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc718102470 0x7fc71819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc715d9b700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc718102e30 0x7fc71819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 -- 172.21.15.99:0/1860856062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc718102e30 msgr2=0x7fc71819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc718102e30 0x7fc71819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 -- 172.21.15.99:0/1860856062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc718102470 msgr2=0x7fc71819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc718102470 0x7fc71819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.617+0000 7fc716d9d700 1 -- 172.21.15.99:0/1860856062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7181a5f60 con 0x7fc7181084b0 2024-05-14T17:13:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.618+0000 7fc716d9d700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc7181a5a20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc710007ee0 tx=0x7fc71000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.618+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71000dbd0 con 0x7fc7181084b0 2024-05-14T17:13:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.618+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7181a6190 con 0x7fc7181084b0 2024-05-14T17:13:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.618+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc71000dd70 con 0x7fc7181084b0 2024-05-14T17:13:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.618+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710005990 con 0x7fc7181084b0 2024-05-14T17:13:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.618+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7181a66d0 con 0x7fc7181084b0 2024-05-14T17:13:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.619+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71806a230 con 0x7fc7181084b0 2024-05-14T17:13:33.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.620+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc71001a020 con 0x7fc7181084b0 2024-05-14T17:13:33.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.621+0000 7fc7077fe700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc7000b29f0 0x7fc7000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:33.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.621+0000 7fc71659c700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc7000b29f0 0x7fc7000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.621+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fc7100e24e0 con 0x7fc7181084b0 2024-05-14T17:13:33.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.621+0000 7fc71659c700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc7000b29f0 0x7fc7000b4eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc70c006010 tx=0x7fc70c01a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.624+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc710010070 con 0x7fc7181084b0 2024-05-14T17:13:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.875+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc71802e720 con 0x7fc7181084b0 2024-05-14T17:13:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.876+0000 7fc7077fe700 1 -- 172.21.15.99:0/1860856062 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11879 (secure 0 0 0) 0x7fc7100a9ec0 con 0x7fc7181084b0 2024-05-14T17:13:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:13:33.876 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:13:10.003659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:13:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc7000b29f0 msgr2=0x7fc7000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc7000b29f0 0x7fc7000b4eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc70c006010 tx=0x7fc70c01a040 comp rx=0 tx=0).stop 2024-05-14T17:13:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 msgr2=0x7fc7181a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc7181a5a20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc710007ee0 tx=0x7fc71000d3b0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 shutdown_connections 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc7000b29f0 0x7fc7000b4eb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7181084b0 0x7fc7181a5a20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc718102470 0x7fc71819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 --2- 172.21.15.99:0/1860856062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc718102e30 0x7fc71819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 >> 172.21.15.99:0/1860856062 conn(0x7fc7180793f0 msgr2=0x7fc7181004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 shutdown_connections 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:33.878+0000 7fc71cf49700 1 -- 172.21.15.99:0/1860856062 wait complete. 2024-05-14T17:13:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 38 2024-05-14T17:13:34.190 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:34 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1860856062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:13:34.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:34 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1860856062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:13:34.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:34 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1860856062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:13:34.519 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 84368c3 --release --await' 2024-05-14T17:13:34.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 84368c3 --release --await 2024-05-14T17:13:34.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.48 MB/sec execute 346 sec latency 18449.291 ms 2024-05-14T17:13:34.768 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:13:35.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:35 smithi099 ceph-mon[31851]: pgmap v311: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 819 KiB/s wr, 48 op/s 2024-05-14T17:13:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:35 smithi193 ceph-mon[35962]: pgmap v311: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 819 KiB/s wr, 48 op/s 2024-05-14T17:13:35.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:35 smithi150 ceph-mon[36204]: pgmap v311: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 819 KiB/s wr, 48 op/s 2024-05-14T17:13:35.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 426911 4.47 MB/sec execute 347 sec latency 19449.318 ms 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 -- 172.21.15.99:0/1282733569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 msgr2=0x7f16f4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 --2- 172.21.15.99:0/1282733569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4110470 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b0d0 tx=0x7f16ec00b3e0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 -- 172.21.15.99:0/1282733569 shutdown_connections 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 --2- 172.21.15.99:0/1282733569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4110470 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 --2- 172.21.15.99:0/1282733569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 0x7f16f4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 --2- 172.21.15.99:0/1282733569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f41041a0 0x7f16f41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 -- 172.21.15.99:0/1282733569 >> 172.21.15.99:0/1282733569 conn(0x7f16f407a6a0 msgr2=0x7f16f407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 -- 172.21.15.99:0/1282733569 shutdown_connections 2024-05-14T17:13:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.331+0000 7f16fc600700 1 -- 172.21.15.99:0/1282733569 wait complete. 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 Processor -- start 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 -- start start 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 0x7f16f419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f41041a0 0x7f16f419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41137b0 con 0x7f16f41088f0 2024-05-14T17:13:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4113630 con 0x7f16f41041a0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.332+0000 7f16fc600700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4113930 con 0x7f16f41037f0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fa39c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 0x7f16f419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fa39c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 0x7f16f419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:55842/0 (socket says 172.21.15.99:55842) 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fa39c700 1 -- 172.21.15.99:0/3359329667 learned_addr learned my addr 172.21.15.99:0/3359329667 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f41041a0 0x7f16f419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 -- 172.21.15.99:0/3359329667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 msgr2=0x7f16f419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 0x7f16f419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 -- 172.21.15.99:0/3359329667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f41041a0 msgr2=0x7f16f419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f41041a0 0x7f16f419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 -- 172.21.15.99:0/3359329667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f419a1b0 con 0x7f16f41088f0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fab9d700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4199a50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009fd0 tx=0x7f16ec007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16eb7fe700 1 -- 172.21.15.99:0/3359329667 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec021070 con 0x7f16f41088f0 2024-05-14T17:13:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f419a440 con 0x7f16f41088f0 2024-05-14T17:13:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f41a6760 con 0x7f16f41088f0 2024-05-14T17:13:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16eb7fe700 1 -- 172.21.15.99:0/3359329667 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ec007e20 con 0x7f16f41088f0 2024-05-14T17:13:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.333+0000 7f16eb7fe700 1 -- 172.21.15.99:0/3359329667 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec02b060 con 0x7f16f41088f0 2024-05-14T17:13:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.336+0000 7f16eb7fe700 1 -- 172.21.15.99:0/3359329667 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f16ec024070 con 0x7f16f41088f0 2024-05-14T17:13:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.336+0000 7f16eb7fe700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b29f0 0x7f16e00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.336+0000 7f16fa39c700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b29f0 0x7f16e00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.336+0000 7f16eb7fe700 1 -- 172.21.15.99:0/3359329667 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f16ec0e35f0 con 0x7f16f41088f0 2024-05-14T17:13:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f16fa39c700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b29f0 0x7f16e00b4eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f16f419ab90 tx=0x7f16f0009290 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 Processor -- start 2024-05-14T17:13:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 -- start start 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfea64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfea6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562acfe04d90 con 0x562acfeabda0 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562acfdf4930 con 0x562acfeac6c0 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f1706e96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562acfdf8970 con 0x562acfea6ad0 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f16fa39c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f16fa39c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37978/0 (socket says 172.21.15.99:37978) 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f16fa39c700 1 -- 172.21.15.99:0/2911111435 learned_addr learned my addr 172.21.15.99:0/2911111435 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfea64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.337+0000 7f16fab9d700 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfea6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16fa39c700 1 -- 172.21.15.99:0/2911111435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 msgr2=0x562acfea6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfea6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16fa39c700 1 -- 172.21.15.99:0/2911111435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 msgr2=0x562acfea64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfea64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16fa39c700 1 -- 172.21.15.99:0/2911111435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562acfd449d0 con 0x562acfeabda0 2024-05-14T17:13:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeac180 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009fd0 tx=0x7f16f0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32e8629fa1e014f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16e8ff9700 1 -- 172.21.15.99:0/2911111435 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f001b040 con 0x562acfeabda0 2024-05-14T17:13:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16e8ff9700 1 -- 172.21.15.99:0/2911111435 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f0024070 con 0x562acfeabda0 2024-05-14T17:13:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.338+0000 7f16e8ff9700 1 -- 172.21.15.99:0/2911111435 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f00137d0 con 0x562acfeabda0 2024-05-14T17:13:36.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 -- 172.21.15.99:0/2911111435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 msgr2=0x562acfeac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeac180 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009fd0 tx=0x7f16f0011000 comp rx=0 tx=0).stop 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 -- 172.21.15.99:0/2911111435 shutdown_connections 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfea6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeac180 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2911111435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfea64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 -- 172.21.15.99:0/2911111435 >> 172.21.15.99:0/2911111435 conn(0x562acfdd1350 msgr2=0x562acfea0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 -- 172.21.15.99:0/2911111435 shutdown_connections 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 -- 172.21.15.99:0/2911111435 wait complete. 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 -- start start 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.339+0000 7f1706e96b80 1 Processor -- start 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeabda0 0x562acfeaf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 0x562acfeafc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562acfcfb0a0 con 0x562acfeac6c0 2024-05-14T17:13:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562acfd968c0 con 0x562acfeabda0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562acfeb3100 con 0x562acfea6ad0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 10 client.0 _ll_register_callbacks cb 0x562acfd8b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 0x562acfeafc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 0x562acfeafc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37994/0 (socket says 172.21.15.99:37994) 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 -- 172.21.15.99:0/1776660913 learned_addr learned my addr 172.21.15.99:0/1776660913 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fa39c700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 -- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 msgr2=0x562acfeb2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeabda0 0x562acfeaf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 -- 172.21.15.99:0/1776660913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeabda0 msgr2=0x562acfeaf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeabda0 0x562acfeaf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 -- 172.21.15.99:0/1776660913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562acfd93790 con 0x562acfeac6c0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16fab9d700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 0x562acfeafc00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0a69e0 tx=0x7f16ec0a61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c33a5d054e38f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16ddffb700 1 -- 172.21.15.99:0/1776660913 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0aa960 con 0x562acfeac6c0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f16ddffb700 1 -- 172.21.15.99:0/1776660913 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f16ec0b10b0 con 0x562acfeac6c0 2024-05-14T17:13:36.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.340+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f0024070 con 0x562acfeac6c0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f16ddffb700 1 -- 172.21.15.99:0/1776660913 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f16ec0aed10 con 0x562acfeac6c0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 10 client.16488 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562acfebd7e0 con 0x562acfeac6c0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f16ddffb700 1 -- 172.21.15.99:0/1776660913 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f0024070 con 0x562acfeac6c0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f16ddffb700 1 -- 172.21.15.99:0/1776660913 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f16ec011070 con 0x562acfeac6c0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 10 client.16488 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 20 client.16488 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acfebfde0 0x562acfec2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 4 client.16488 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562acfe72460 con 0x562acfebfde0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.341+0000 7f16fa39c700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acfebfde0 0x562acfec2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.342+0000 7f16fa39c700 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acfebfde0 0x562acfec2200 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3789fedfab7b9ab server_cookie=fa633ca3a8bb4661 in_seq=0 out_seq=0 2024-05-14T17:13:36.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.342+0000 7f16ddffb700 10 client.16488 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:36.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.342+0000 7f16ddffb700 1 -- 172.21.15.99:0/1776660913 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562acfe72460 con 0x562acfebfde0 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.342+0000 7f16ddffb700 10 client.16488 handle_command_reply: tid=0 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.342+0000 7f1706e96b80 1 client.16488 shutdown 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 msgr2=0x562acfeafc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 0x562acfeafc00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0a69e0 tx=0x7f16ec0a61e0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 shutdown_connections 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acfebfde0 0x562acfec2200 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeac6c0 0x562acfeafc00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 --2- 172.21.15.99:0/1776660913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeabda0 0x562acfeaf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 >> 172.21.15.99:0/1776660913 conn(0x562acfdd1350 msgr2=0x562acfe9b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f16ddffb700 0 client.16488 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 shutdown_connections 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 1 -- 172.21.15.99:0/1776660913 wait complete. 2024-05-14T17:13:36.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.343+0000 7f1706e96b80 20 client.16488 trim_cache size 0 max 16384 2024-05-14T17:13:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 Processor -- start 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 -- start start 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 0x562acfebc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfeabda0 0x562acfe9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeaaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16d800ff10 con 0x562acfea6ad0 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16d800d6e0 con 0x562acfeac6c0 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f1706e96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16d800b550 con 0x562acfeabda0 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f16fa39c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 0x562acfebc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f16fa39c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 0x562acfebc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37996/0 (socket says 172.21.15.99:37996) 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.362+0000 7f16fa39c700 1 -- 172.21.15.99:0/2190889419 learned_addr learned my addr 172.21.15.99:0/2190889419 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fab9d700 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeaaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfeabda0 0x562acfe9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fa39c700 1 -- 172.21.15.99:0/2190889419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfeabda0 msgr2=0x562acfe9b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfeabda0 0x562acfe9b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fa39c700 1 -- 172.21.15.99:0/2190889419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 msgr2=0x562acfeaaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeaaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fa39c700 1 -- 172.21.15.99:0/2190889419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562acfeadc30 con 0x562acfea6ad0 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 0x562acfebc070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f16f0004790 tx=0x7f16f000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc1c21d005ac9cf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16dffff700 1 -- 172.21.15.99:0/2190889419 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0030070 con 0x562acfea6ad0 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16dffff700 1 -- 172.21.15.99:0/2190889419 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f0024040 con 0x562acfea6ad0 2024-05-14T17:13:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.363+0000 7f16dffff700 1 -- 172.21.15.99:0/2190889419 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0015af0 con 0x562acfea6ad0 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 -- 172.21.15.99:0/2190889419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 msgr2=0x562acfebc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 0x562acfebc070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f16f0004790 tx=0x7f16f000eef0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 -- 172.21.15.99:0/2190889419 shutdown_connections 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeaaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfea6ad0 0x562acfebc070 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2190889419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfeabda0 0x562acfe9b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 -- 172.21.15.99:0/2190889419 >> 172.21.15.99:0/2190889419 conn(0x562acfdd1350 msgr2=0x562acfeaa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 -- 172.21.15.99:0/2190889419 shutdown_connections 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 -- 172.21.15.99:0/2190889419 wait complete. 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 -- start start 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.364+0000 7f1706e96b80 1 Processor -- start 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeb53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeb5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16d800bba0 con 0x562acfeabda0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16d800d010 con 0x562acfeac6c0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16d800c270 con 0x562acfea6ad0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 10 client.0 _ll_register_callbacks cb 0x562acfd8b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeb53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeb53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38000/0 (socket says 172.21.15.99:38000) 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 -- 172.21.15.99:0/2513947478 learned_addr learned my addr 172.21.15.99:0/2513947478 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16fab9d700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeb5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 -- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 msgr2=0x562acfeb89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 -- 172.21.15.99:0/2513947478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 msgr2=0x562acfeb5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeb5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 -- 172.21.15.99:0/2513947478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562acfeb61d0 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16f9b9b700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeb53c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x562acfeacac0 tx=0x7f16e4015640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eae9e7019a91f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16ddffb700 1 -- 172.21.15.99:0/2513947478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e401b070 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f16ddffb700 1 -- 172.21.15.99:0/2513947478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f16e4015db0 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.365+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f0024040 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f16ddffb700 1 -- 172.21.15.99:0/2513947478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f16e4020460 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 10 client.16494 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562acff2b850 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f16ddffb700 1 -- 172.21.15.99:0/2513947478 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f0024040 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f16ddffb700 1 -- 172.21.15.99:0/2513947478 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f16e4017760 con 0x562acfeabda0 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 10 client.16494 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 20 client.16494 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acff2c020 0x562acff2e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:36.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 4 client.16494 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "84368c3", "release": true, "await": true}] 2024-05-14T17:13:36.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "84368c3", "release": true, "await": true}) -- 0x562acfe06ce0 con 0x562acff2c020 2024-05-14T17:13:36.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.366+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acff2c020 0x562acff2e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.367+0000 7f16fa39c700 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acff2c020 0x562acff2e440 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1939a485ca8aa9 server_cookie=953c87281caaee2f in_seq=0 out_seq=0 2024-05-14T17:13:36.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.367+0000 7f16ddffb700 10 client.16494 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f16ddffb700 1 -- 172.21.15.99:0/2513947478 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562acfe06ce0 con 0x562acff2c020 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f16ddffb700 10 client.16494 handle_command_reply: tid=0 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 client.16494 shutdown 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 msgr2=0x562acfeb53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeb53c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x562acfeacac0 tx=0x7f16e4015640 comp rx=0 tx=0).stop 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 shutdown_connections 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562acfeac6c0 0x562acfeb5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562acfea6ad0 0x562acfeb89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x562acff2c020 0x562acff2e440 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 --2- 172.21.15.99:0/2513947478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562acfeabda0 0x562acfeb53c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f16ddffb700 0 client.16494 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:36.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 >> 172.21.15.99:0/2513947478 conn(0x562acfdd1350 msgr2=0x562acfeab050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 shutdown_connections 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 1 -- 172.21.15.99:0/2513947478 wait complete. 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.372+0000 7f1706e96b80 20 client.16494 trim_cache size 0 max 16384 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 38, 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 32, 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "84368c3": { 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 32, 2024-05-14T17:13:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 140.4, 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b29f0 msgr2=0x7f16e00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b29f0 0x7f16e00b4eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f16f419ab90 tx=0x7f16f0009290 comp rx=0 tx=0).stop 2024-05-14T17:13:36.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 msgr2=0x7f16f4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4199a50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009fd0 tx=0x7f16ec007a20 comp rx=0 tx=0).stop 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 shutdown_connections 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f16e00b29f0 0x7f16e00b4eb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16f41088f0 0x7f16f4199a50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16f41037f0 0x7f16f419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 --2- 172.21.15.99:0/3359329667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16f41041a0 0x7f16f419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 >> 172.21.15.99:0/3359329667 conn(0x7f16f407a6a0 msgr2=0x7f16f4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 shutdown_connections 2024-05-14T17:13:36.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:13:36.373+0000 7f16fc600700 1 -- 172.21.15.99:0/3359329667 wait complete. 2024-05-14T17:13:36.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427203 4.46 MB/sec execute 348 sec latency 20280.932 ms 2024-05-14T17:13:36.900 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '84368c3', total seconds elapsed: 25.6 2024-05-14T17:13:37.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:37 smithi099 ceph-mon[31851]: pgmap v312: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 550 KiB/s wr, 34 op/s 2024-05-14T17:13:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:37 smithi193 ceph-mon[35962]: pgmap v312: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 550 KiB/s wr, 34 op/s 2024-05-14T17:13:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:37 smithi150 ceph-mon[36204]: pgmap v312: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 550 KiB/s wr, 34 op/s 2024-05-14T17:13:37.547 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 4.45 MB/sec execute 349 sec latency 901.567 ms 2024-05-14T17:13:38.548 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 4.44 MB/sec execute 350 sec latency 1901.683 ms 2024-05-14T17:13:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:39 smithi193 ceph-mon[35962]: pgmap v313: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 550 KiB/s wr, 34 op/s 2024-05-14T17:13:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:39 smithi150 ceph-mon[36204]: pgmap v313: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 550 KiB/s wr, 34 op/s 2024-05-14T17:13:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:39 smithi099 ceph-mon[31851]: pgmap v313: 97 pgs: 97 active+clean; 110 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 550 KiB/s wr, 34 op/s 2024-05-14T17:13:39.548 INFO:tasks.workunit.client.0.smithi099.stdout: 1 427365 4.43 MB/sec execute 351 sec latency 2901.805 ms 2024-05-14T17:13:40.548 INFO:tasks.workunit.client.0.smithi099.stdout: 1 429531 4.45 MB/sec execute 352 sec latency 2957.925 ms 2024-05-14T17:13:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:41 smithi193 ceph-mon[35962]: pgmap v314: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.1 MiB/s wr, 59 op/s 2024-05-14T17:13:41.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:41 smithi150 ceph-mon[36204]: pgmap v314: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.1 MiB/s wr, 59 op/s 2024-05-14T17:13:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:41 smithi099 ceph-mon[31851]: pgmap v314: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.1 MiB/s wr, 59 op/s 2024-05-14T17:13:41.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 4.46 MB/sec execute 353 sec latency 48.625 ms 2024-05-14T17:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 -- 172.21.15.99:0/4074772926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d41011c0 msgr2=0x7f91d41015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 --2- 172.21.15.99:0/4074772926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d41011c0 0x7f91d41015a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009900 tx=0x7f91c802a410 comp rx=0 tx=0).stop 2024-05-14T17:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 -- 172.21.15.99:0/4074772926 shutdown_connections 2024-05-14T17:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 --2- 172.21.15.99:0/4074772926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d4107e30 0x7f91d410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 --2- 172.21.15.99:0/4074772926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91d4101b70 0x7f91d4107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 --2- 172.21.15.99:0/4074772926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d41011c0 0x7f91d41015a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 -- 172.21.15.99:0/4074772926 >> 172.21.15.99:0/4074772926 conn(0x7f91d4077eb0 msgr2=0x7f91d40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 -- 172.21.15.99:0/4074772926 shutdown_connections 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.497+0000 7f91dab38640 1 -- 172.21.15.99:0/4074772926 wait complete. 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 Processor -- start 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 -- start start 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d41011c0 0x7f91d41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91d4107e30 0x7f91d4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 0x7f91d419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91d4112d90 con 0x7f91d419da90 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91d4112c10 con 0x7f91d4107e30 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91dab38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91d4112f10 con 0x7f91d41011c0 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91d90d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 0x7f91d419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91d90d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 0x7f91d419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38026/0 (socket says 172.21.15.99:38026) 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91d90d5640 1 -- 172.21.15.99:0/3434700387 learned_addr learned my addr 172.21.15.99:0/3434700387 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91d4107e30 0x7f91d4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.498+0000 7f91d88d4640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d41011c0 0x7f91d41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d90d5640 1 -- 172.21.15.99:0/3434700387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d41011c0 msgr2=0x7f91d41991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d90d5640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d41011c0 0x7f91d41991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d90d5640 1 -- 172.21.15.99:0/3434700387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91d4107e30 msgr2=0x7f91d4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d90d5640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91d4107e30 0x7f91d4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d90d5640 1 -- 172.21.15.99:0/3434700387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d419e580 con 0x7f91d419da90 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d90d5640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 0x7f91d419df40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00cad0 tx=0x7f91cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d1ffb640 1 -- 172.21.15.99:0/3434700387 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00ae20 con 0x7f91d419da90 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d1ffb640 1 -- 172.21.15.99:0/3434700387 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91cc005020 con 0x7f91d419da90 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d419e870 con 0x7f91d419da90 2024-05-14T17:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91d1ffb640 1 -- 172.21.15.99:0/3434700387 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00f6b0 con 0x7f91d419da90 2024-05-14T17:13:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.499+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d41a6060 con 0x7f91d419da90 2024-05-14T17:13:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.501+0000 7f91d1ffb640 1 -- 172.21.15.99:0/3434700387 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f91cc0051d0 con 0x7f91d419da90 2024-05-14T17:13:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 Processor -- start 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 -- start start 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb279bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb2889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb28b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbbb1ecc30 con 0x55bbbb288ee0 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbbb1dc7d0 con 0x55bbbb27a0f0 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91dd855740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bbbb1e0810 con 0x55bbbb2797d0 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d1ffb640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f91c40b2a90 0x7f91c40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb28b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb28b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38030/0 (socket says 172.21.15.99:38030) 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d3fff640 1 -- 172.21.15.99:0/3270263713 learned_addr learned my addr 172.21.15.99:0/3270263713 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d88d4640 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb2889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb279bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.502+0000 7f91d90d5640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f91c40b2a90 0x7f91c40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d3fff640 1 -- 172.21.15.99:0/3270263713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 msgr2=0x55bbbb279bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb279bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d1ffb640 1 -- 172.21.15.99:0/3434700387 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f91cc0e24c0 con 0x7f91d419da90 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d3fff640 1 -- 172.21.15.99:0/3270263713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 msgr2=0x55bbbb2889a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb2889a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d3fff640 1 -- 172.21.15.99:0/3270263713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbbaef1210 con 0x55bbbb288ee0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d88d4640 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb2889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb28b2d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f91c0012750 tx=0x7f91c0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a01c6118cf888d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91d90d5640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f91c40b2a90 0x7f91c40b4f50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0dfcc0 tx=0x7f91cc076180 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91beffd640 1 -- 172.21.15.99:0/3270263713 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0012cd0 con 0x55bbbb288ee0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91beffd640 1 -- 172.21.15.99:0/3270263713 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0005020 con 0x55bbbb288ee0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.503+0000 7f91beffd640 1 -- 172.21.15.99:0/3270263713 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0004a00 con 0x55bbbb288ee0 2024-05-14T17:13:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 -- 172.21.15.99:0/3270263713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 msgr2=0x55bbbb28b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb28b2d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f91c0012750 tx=0x7f91c0005c90 comp rx=0 tx=0).stop 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 -- 172.21.15.99:0/3270263713 shutdown_connections 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb28b2d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb2889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 --2- 172.21.15.99:0/3270263713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb279bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 -- 172.21.15.99:0/3270263713 >> 172.21.15.99:0/3270263713 conn(0x55bbbb275e00 msgr2=0x55bbbb2761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 -- 172.21.15.99:0/3270263713 shutdown_connections 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 -- 172.21.15.99:0/3270263713 wait complete. 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 -- start start 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 Processor -- start 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.504+0000 7f91dd855740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb288910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb27a0f0 0x55bbbb285c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb286340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbbaec2790 con 0x55bbbb288ee0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbbb198890 con 0x55bbbb2797d0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bbbb28bbb0 con 0x55bbbb27a0f0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 10 client.0 _ll_register_callbacks cb 0x55bbbb19bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb286340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb286340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38038/0 (socket says 172.21.15.99:38038) 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 -- 172.21.15.99:0/504343873 learned_addr learned my addr 172.21.15.99:0/504343873 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d88d4640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb27a0f0 0x55bbbb285c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb288910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 -- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb27a0f0 msgr2=0x55bbbb285c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb27a0f0 0x55bbbb285c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 -- 172.21.15.99:0/504343873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 msgr2=0x55bbbb288910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb288910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 -- 172.21.15.99:0/504343873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbbafbd620 con 0x55bbbb288ee0 2024-05-14T17:13:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91d3fff640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb286340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f91c0012d40 tx=0x7f91c001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8106dd6fcf77ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91b3fff640 1 -- 172.21.15.99:0/504343873 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0022070 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91b3fff640 1 -- 172.21.15.99:0/504343873 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f91c0005400 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c0005020 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91b3fff640 1 -- 172.21.15.99:0/504343873 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f91c0034050 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 10 client.16506 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.505+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bbbb297700 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91b3fff640 1 -- 172.21.15.99:0/504343873 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c0005020 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91b3fff640 1 -- 172.21.15.99:0/504343873 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f91c0004840 con 0x55bbbb288ee0 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91dd855740 10 client.16506 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91dd855740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91dd855740 20 client.16506 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91dd855740 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb29af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91dd855740 4 client.16506 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbbb1eeb80 con 0x55bbbb298b10 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.506+0000 7f91d88d4640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb29af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.507+0000 7f91b3fff640 10 client.16506 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.507+0000 7f91d88d4640 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb29af00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457939389d53c1b1 server_cookie=a325b86376452d8 in_seq=0 out_seq=0 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.507+0000 7f91b3fff640 1 -- 172.21.15.99:0/504343873 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbbb1eeb80 con 0x55bbbb298b10 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.507+0000 7f91b3fff640 10 client.16506 handle_command_reply: tid=0 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.507+0000 7f91dd855740 1 client.16506 shutdown 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 msgr2=0x55bbbb286340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb286340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f91c0012d40 tx=0x7f91c001b040 comp rx=0 tx=0).stop 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 shutdown_connections 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb29af00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb286340 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91b3fff640 0 client.16506 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb27a0f0 0x55bbbb285c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 --2- 172.21.15.99:0/504343873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb288910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 >> 172.21.15.99:0/504343873 conn(0x55bbbb275e00 msgr2=0x55bbbb27bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 shutdown_connections 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 1 -- 172.21.15.99:0/504343873 wait complete. 2024-05-14T17:13:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.508+0000 7f91dd855740 20 client.16506 trim_cache size 0 max 16384 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 Processor -- start 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 -- start start 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb270f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb294ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb288440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41aed90 con 0x55bbbb288ee0 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41ac5a0 con 0x55bbbb27a0f0 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91dd855740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41aa410 con 0x55bbbb2797d0 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb288440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb288440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38048/0 (socket says 172.21.15.99:38048) 2024-05-14T17:13:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 -- 172.21.15.99:0/309334878 learned_addr learned my addr 172.21.15.99:0/309334878 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 -- 172.21.15.99:0/309334878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 msgr2=0x55bbbb270f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb270f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 -- 172.21.15.99:0/309334878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 msgr2=0x55bbbb294ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb294ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 -- 172.21.15.99:0/309334878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbbb279ee0 con 0x55bbbb288ee0 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91d3fff640 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb288440 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55bbbb276ee0 tx=0x7f91c0012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b650e9229ab2d3ac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91be7fc640 1 -- 172.21.15.99:0/309334878 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0022040 con 0x55bbbb288ee0 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91be7fc640 1 -- 172.21.15.99:0/309334878 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c000ba60 con 0x55bbbb288ee0 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.523+0000 7f91be7fc640 1 -- 172.21.15.99:0/309334878 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c00109a0 con 0x55bbbb288ee0 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 -- 172.21.15.99:0/309334878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 msgr2=0x55bbbb288440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb288440 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55bbbb276ee0 tx=0x7f91c0012d70 comp rx=0 tx=0).stop 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 -- 172.21.15.99:0/309334878 shutdown_connections 2024-05-14T17:13:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb288ee0 0x55bbbb288440 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb27a0f0 0x55bbbb294ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 --2- 172.21.15.99:0/309334878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb2797d0 0x55bbbb270f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 -- 172.21.15.99:0/309334878 >> 172.21.15.99:0/309334878 conn(0x55bbbb275e00 msgr2=0x55bbbb284c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 -- 172.21.15.99:0/309334878 shutdown_connections 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 -- 172.21.15.99:0/309334878 wait complete. 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 -- start start 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.524+0000 7f91dd855740 1 Processor -- start 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb291150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 0x55bbbb28c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb288ee0 0x55bbbb28ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41aaa60 con 0x55bbbb27a0f0 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41abed0 con 0x55bbbb2797d0 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91d41ab130 con 0x55bbbb288ee0 2024-05-14T17:13:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 10 client.0 _ll_register_callbacks cb 0x7f91c0012650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 0x55bbbb28c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 0x55bbbb28c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38052/0 (socket says 172.21.15.99:38052) 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 -- 172.21.15.99:0/3486910785 learned_addr learned my addr 172.21.15.99:0/3486910785 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb291150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb288ee0 0x55bbbb28ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 -- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb288ee0 msgr2=0x55bbbb28ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb288ee0 0x55bbbb28ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 -- 172.21.15.99:0/3486910785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 msgr2=0x55bbbb291150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb291150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 -- 172.21.15.99:0/3486910785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbbb28cfc0 con 0x55bbbb27a0f0 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb288ee0 0x55bbbb28ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91d88d4640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 0x55bbbb28c4c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009900 tx=0x7f91c8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e05c3a400c8d009 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91b3fff640 1 -- 172.21.15.99:0/3486910785 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8035040 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91b3fff640 1 -- 172.21.15.99:0/3486910785 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f91c8005590 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbbb1e8360 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.525+0000 7f91b3fff640 1 -- 172.21.15.99:0/3486910785 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f91c8054050 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 10 client.16512 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bbbb302fe0 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91b3fff640 1 -- 172.21.15.99:0/3486910785 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bbbb1e8360 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91b3fff640 1 -- 172.21.15.99:0/3486910785 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f91c80468f0 con 0x55bbbb27a0f0 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 10 client.16512 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 20 client.16512 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb3057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 4 client.16512 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbbb1625f0 con 0x55bbbb298b10 2024-05-14T17:13:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb3057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91b3fff640 10 client.16512 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.526+0000 7f91d3fff640 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb3057d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b834c5b0408fdf4 server_cookie=df07d9647c86b9d0 in_seq=0 out_seq=0 2024-05-14T17:13:42.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 4.44 MB/sec execute 354 sec latency 1049.631 ms 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91b3fff640 1 -- 172.21.15.99:0/3486910785 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bbbb1625f0 con 0x55bbbb298b10 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91b3fff640 10 client.16512 handle_command_reply: tid=0 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 client.16512 shutdown 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 msgr2=0x55bbbb28c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 0x55bbbb28c4c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009900 tx=0x7f91c8005c00 comp rx=0 tx=0).stop 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 shutdown_connections 2024-05-14T17:13:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbb298b10 0x55bbbb3057d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbb288ee0 0x55bbbb28ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91b3fff640 0 client.16512 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbb27a0f0 0x55bbbb28c4c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 --2- 172.21.15.99:0/3486910785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbb2797d0 0x55bbbb291150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 >> 172.21.15.99:0/3486910785 conn(0x55bbbb275e00 msgr2=0x55bbbb284c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 shutdown_connections 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 1 -- 172.21.15.99:0/3486910785 wait complete. 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.577+0000 7f91dd855740 20 client.16512 trim_cache size 0 max 16384 2024-05-14T17:13:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:13:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:13:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:13:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f91c40b2a90 msgr2=0x7f91c40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f91c40b2a90 0x7f91c40b4f50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0dfcc0 tx=0x7f91cc076180 comp rx=0 tx=0).stop 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 msgr2=0x7f91d419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 0x7f91d419df40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00cad0 tx=0x7f91cc00cf90 comp rx=0 tx=0).stop 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 shutdown_connections 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f91c40b2a90 0x7f91c40b4f50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91d419da90 0x7f91d419df40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91d4107e30 0x7f91d4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 --2- 172.21.15.99:0/3434700387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d41011c0 0x7f91d41991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 >> 172.21.15.99:0/3434700387 conn(0x7f91d4077eb0 msgr2=0x7f91d4105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 shutdown_connections 2024-05-14T17:13:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:42.578+0000 7f91dab38640 1 -- 172.21.15.99:0/3434700387 wait complete. 2024-05-14T17:13:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:43 smithi193 ceph-mon[35962]: pgmap v315: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 811 KiB/s wr, 44 op/s 2024-05-14T17:13:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:43 smithi150 ceph-mon[36204]: pgmap v315: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 811 KiB/s wr, 44 op/s 2024-05-14T17:13:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:43 smithi099 ceph-mon[31851]: pgmap v315: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 811 KiB/s wr, 44 op/s 2024-05-14T17:13:43.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 4.43 MB/sec execute 355 sec latency 2049.719 ms 2024-05-14T17:13:44.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 430914 4.42 MB/sec execute 356 sec latency 3049.802 ms 2024-05-14T17:13:45.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:45 smithi150 ceph-mon[36204]: pgmap v316: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-05-14T17:13:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:45 smithi099 ceph-mon[31851]: pgmap v316: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-05-14T17:13:45.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:45 smithi193 ceph-mon[35962]: pgmap v316: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-05-14T17:13:45.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 433182 4.44 MB/sec execute 357 sec latency 3069.655 ms 2024-05-14T17:13:46.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 4.45 MB/sec execute 358 sec latency 189.020 ms 2024-05-14T17:13:47.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:47 smithi150 ceph-mon[36204]: pgmap v317: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-14T17:13:47.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:47 smithi099 ceph-mon[31851]: pgmap v317: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-14T17:13:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:47 smithi193 ceph-mon[35962]: pgmap v317: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-14T17:13:47.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 4.43 MB/sec execute 359 sec latency 1189.103 ms 2024-05-14T17:13:48.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 4.42 MB/sec execute 360 sec latency 2189.250 ms 2024-05-14T17:13:49.316 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:49 smithi099 ceph-mon[31851]: pgmap v318: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-14T17:13:49.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:49 smithi150 ceph-mon[36204]: pgmap v318: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-14T17:13:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:49 smithi193 ceph-mon[35962]: pgmap v318: 97 pgs: 97 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-14T17:13:49.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 434458 4.41 MB/sec execute 361 sec latency 3189.300 ms 2024-05-14T17:13:50.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 436719 4.44 MB/sec execute 362 sec latency 3239.775 ms 2024-05-14T17:13:51.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:51 smithi150 ceph-mon[36204]: pgmap v319: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-05-14T17:13:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:51 smithi099 ceph-mon[31851]: pgmap v319: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-05-14T17:13:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:51 smithi193 ceph-mon[35962]: pgmap v319: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-05-14T17:13:51.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 4.44 MB/sec execute 363 sec latency 190.319 ms 2024-05-14T17:13:52.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 4.43 MB/sec execute 364 sec latency 1190.401 ms 2024-05-14T17:13:53.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:53 smithi150 ceph-mon[36204]: pgmap v320: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-05-14T17:13:53.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:53 smithi099 ceph-mon[31851]: pgmap v320: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-05-14T17:13:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:53 smithi193 ceph-mon[35962]: pgmap v320: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-05-14T17:13:53.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 4.41 MB/sec execute 365 sec latency 2190.456 ms 2024-05-14T17:13:54.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 438005 4.40 MB/sec execute 366 sec latency 3190.567 ms 2024-05-14T17:13:55.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:55 smithi150 ceph-mon[36204]: pgmap v321: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-14T17:13:55.513 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:55 smithi099 ceph-mon[31851]: pgmap v321: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-14T17:13:55.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:55 smithi193 ceph-mon[35962]: pgmap v321: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-05-14T17:13:55.549 INFO:tasks.workunit.client.0.smithi099.stdout: 1 440312 4.43 MB/sec execute 367 sec latency 3240.326 ms 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 -- 172.21.15.99:0/3695083328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 msgr2=0x7f18b81040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/3695083328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81040e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009900 tx=0x7f18a802a480 comp rx=0 tx=0).stop 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 -- 172.21.15.99:0/3695083328 shutdown_connections 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/3695083328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18b810d0d0 0x7f18b810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/3695083328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b8104620 0x7f18b810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/3695083328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81040e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 -- 172.21.15.99:0/3695083328 >> 172.21.15.99:0/3695083328 conn(0x7f18b8077cd0 msgr2=0x7f18b80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 -- 172.21.15.99:0/3695083328 shutdown_connections 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.670+0000 7f18bdbe5640 1 -- 172.21.15.99:0/3695083328 wait complete. 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 Processor -- start 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 -- start start 2024-05-14T17:13:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b8104620 0x7f18b8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18b810d0d0 0x7f18b819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8112d10 con 0x7f18b8103d00 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8112b90 con 0x7f18b8104620 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18bdbe5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8112e90 con 0x7f18b810d0d0 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38236/0 (socket says 172.21.15.99:38236) 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18b77fe640 1 -- 172.21.15.99:0/2747226654 learned_addr learned my addr 172.21.15.99:0/2747226654 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18affff640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b8104620 0x7f18b8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.671+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18b810d0d0 0x7f18b819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b77fe640 1 -- 172.21.15.99:0/2747226654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18b810d0d0 msgr2=0x7f18b819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18b810d0d0 0x7f18b819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b77fe640 1 -- 172.21.15.99:0/2747226654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b8104620 msgr2=0x7f18b8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b8104620 0x7f18b8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b77fe640 1 -- 172.21.15.99:0/2747226654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b819e190 con 0x7f18b8103d00 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81991f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005940 tx=0x7f18a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b57fa640 1 -- 172.21.15.99:0/2747226654 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a803f030 con 0x7f18b8103d00 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b57fa640 1 -- 172.21.15.99:0/2747226654 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a80044c0 con 0x7f18b8103d00 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b819e420 con 0x7f18b8103d00 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18b57fa640 1 -- 172.21.15.99:0/2747226654 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8035070 con 0x7f18b8103d00 2024-05-14T17:13:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.672+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a5d10 con 0x7f18b8103d00 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.674+0000 7f18b57fa640 1 -- 172.21.15.99:0/2747226654 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f18a8042030 con 0x7f18b8103d00 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 Processor -- start 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 -- start start 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 0x55f86e3a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f86e31cc30 con 0x55f86e3aa210 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f86e30c7d0 con 0x55f86e3a98f0 2024-05-14T17:13:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18c09d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f86e310810 con 0x55f86e3b4ad0 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 0x55f86e3a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 0x55f86e3a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:33764/0 (socket says 172.21.15.99:33764) 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 -- 172.21.15.99:0/527130052 learned_addr learned my addr 172.21.15.99:0/527130052 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18affff640 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b57fa640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f18a40b2a10 0x7f18a40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 -- 172.21.15.99:0/527130052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3b4ad0 msgr2=0x55f86e3b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 -- 172.21.15.99:0/527130052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 msgr2=0x55f86e3b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.675+0000 7f18b7fff640 1 -- 172.21.15.99:0/527130052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f86e021210 con 0x55f86e3a98f0 2024-05-14T17:13:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f18a40b2a10 0x7f18a40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18b57fa640 1 -- 172.21.15.99:0/2747226654 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f18a8102930 con 0x7f18b8103d00 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18b7fff640 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 0x55f86e3a9cd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18b00114c0 tx=0x7f18b00118c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23f3a7741347f03 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18adffb640 1 -- 172.21.15.99:0/527130052 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0017070 con 0x55f86e3a98f0 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f18a40b2a10 0x7f18a40b4ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f18b0022000 tx=0x7f18b0023000 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18adffb640 1 -- 172.21.15.99:0/527130052 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b000fc50 con 0x55f86e3a98f0 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.676+0000 7f18adffb640 1 -- 172.21.15.99:0/527130052 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b001bb30 con 0x55f86e3a98f0 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 -- 172.21.15.99:0/527130052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 msgr2=0x55f86e3a9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 0x55f86e3a9cd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18b00114c0 tx=0x7f18b00118c0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 -- 172.21.15.99:0/527130052 shutdown_connections 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 --2- 172.21.15.99:0/527130052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3a98f0 0x55f86e3a9cd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 -- 172.21.15.99:0/527130052 >> 172.21.15.99:0/527130052 conn(0x55f86e3a5e00 msgr2=0x55f86e3a61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 -- 172.21.15.99:0/527130052 shutdown_connections 2024-05-14T17:13:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 -- 172.21.15.99:0/527130052 wait complete. 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 -- start start 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 Processor -- start 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.677+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f86dff2790 con 0x55f86e3aa210 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f86e2c8890 con 0x55f86e3b4ad0 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f86e3b77a0 con 0x55f86e3a98f0 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 10 client.0 _ll_register_callbacks cb 0x55f86e2cbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38258/0 (socket says 172.21.15.99:38258) 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 -- 172.21.15.99:0/2200181392 learned_addr learned my addr 172.21.15.99:0/2200181392 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 -- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 msgr2=0x55f86e3b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 -- 172.21.15.99:0/2200181392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 msgr2=0x55f86e3b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 -- 172.21.15.99:0/2200181392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f86e0ed620 con 0x55f86e3aa210 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18affff640 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b1820 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f18a0012700 tx=0x7f18a0012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70a3c0a85b17311 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f1896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f1896ffd640 1 -- 172.21.15.99:0/2200181392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a000f5e0 con 0x55f86e3aa210 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f1896ffd640 1 -- 172.21.15.99:0/2200181392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f18a000f780 con 0x55f86e3aa210 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b000fc50 con 0x55f86e3aa210 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f1896ffd640 1 -- 172.21.15.99:0/2200181392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f18a000f980 con 0x55f86e3aa210 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 10 client.16524 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.678+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f86e3c3320 con 0x55f86e3aa210 2024-05-14T17:13:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f1896ffd640 1 -- 172.21.15.99:0/2200181392 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b000fc50 con 0x55f86e3aa210 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f1896ffd640 1 -- 172.21.15.99:0/2200181392 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f18a0010bf0 con 0x55f86e3aa210 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18c09d6740 10 client.16524 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18c09d6740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18c09d6740 20 client.16524 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e3c4920 0x55f86e3c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18c09d6740 4 client.16524 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f86e31eb80 con 0x55f86e3c4920 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e3c4920 0x55f86e3c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e3c4920 0x55f86e3c6d60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ca5a350ca9eb9f server_cookie=f6aa99dc1a0fdcbf in_seq=0 out_seq=0 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.679+0000 7f1896ffd640 10 client.16524 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f1896ffd640 1 -- 172.21.15.99:0/2200181392 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f86e31eb80 con 0x55f86e3c4920 2024-05-14T17:13:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f1896ffd640 10 client.16524 handle_command_reply: tid=0 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 client.16524 shutdown 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 msgr2=0x55f86e3b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b1820 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f18a0012700 tx=0x7f18a0012b00 comp rx=0 tx=0).stop 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 shutdown_connections 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e3c4920 0x55f86e3c6d60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3aa210 0x55f86e3b1820 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f1896ffd640 0 client.16524 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2200181392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 >> 172.21.15.99:0/2200181392 conn(0x55f86e3a5e00 msgr2=0x55f86e3a5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 shutdown_connections 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 1 -- 172.21.15.99:0/2200181392 wait complete. 2024-05-14T17:13:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.680+0000 7f18c09d6740 20 client.16524 trim_cache size 0 max 16384 2024-05-14T17:13:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.696+0000 7f18c09d6740 1 Processor -- start 2024-05-14T17:13:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.696+0000 7f18c09d6740 1 -- start start 2024-05-14T17:13:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 0x55f86e3b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3aa210 0x55f86e3b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18c09d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81b0f80 con 0x55f86e3a98f0 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18c09d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81ae790 con 0x55f86e3b4ad0 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18c09d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81ac600 con 0x55f86e3aa210 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 0x55f86e3b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 0x55f86e3b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38266/0 (socket says 172.21.15.99:38266) 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 -- 172.21.15.99:0/120355224 learned_addr learned my addr 172.21.15.99:0/120355224 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b7fff640 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18affff640 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3aa210 0x55f86e3b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 -- 172.21.15.99:0/120355224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3aa210 msgr2=0x55f86e3b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3aa210 0x55f86e3b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 -- 172.21.15.99:0/120355224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 msgr2=0x55f86e3cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 -- 172.21.15.99:0/120355224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f86e3aa040 con 0x55f86e3a98f0 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18b77fe640 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 0x55f86e3b0a90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f18a8006010 tx=0x7f18a8104e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fe4686cf99eccf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18ad7fa640 1 -- 172.21.15.99:0/120355224 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8108020 con 0x55f86e3a98f0 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18ad7fa640 1 -- 172.21.15.99:0/120355224 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a80d23b0 con 0x55f86e3a98f0 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.697+0000 7f18ad7fa640 1 -- 172.21.15.99:0/120355224 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8049040 con 0x55f86e3a98f0 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 -- 172.21.15.99:0/120355224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 msgr2=0x55f86e3b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 0x55f86e3b0a90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f18a8006010 tx=0x7f18a8104e40 comp rx=0 tx=0).stop 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 -- 172.21.15.99:0/120355224 shutdown_connections 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3b4ad0 0x55f86e3cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3aa210 0x55f86e3b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 --2- 172.21.15.99:0/120355224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3a98f0 0x55f86e3b0a90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 -- 172.21.15.99:0/120355224 >> 172.21.15.99:0/120355224 conn(0x55f86e3a5e00 msgr2=0x55f86e3acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 -- 172.21.15.99:0/120355224 shutdown_connections 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 -- 172.21.15.99:0/120355224 wait complete. 2024-05-14T17:13:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.698+0000 7f18c09d6740 1 -- start start 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 Processor -- start 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3aa210 0x55f86e3aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 0x55f86e3af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81acc50 con 0x55f86e3b4ad0 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81ae0c0 con 0x55f86e3aa210 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81ad320 con 0x55f86e3a98f0 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18c09d6740 10 client.0 _ll_register_callbacks cb 0x7f18a0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 0x55f86e3af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 0x55f86e3af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38270/0 (socket says 172.21.15.99:38270) 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18affff640 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3aa210 0x55f86e3aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 -- 172.21.15.99:0/2260806531 learned_addr learned my addr 172.21.15.99:0/2260806531 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 -- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 msgr2=0x55f86e3ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 -- 172.21.15.99:0/2260806531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3aa210 msgr2=0x55f86e3aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3aa210 0x55f86e3aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.699+0000 7f18b7fff640 1 -- 172.21.15.99:0/2260806531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f86e42e1e0 con 0x55f86e3b4ad0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18b7fff640 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 0x55f86e3af0c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f18b0011460 tx=0x7f18b0003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b058cf746bd75741 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f1896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f1896ffd640 1 -- 172.21.15.99:0/2260806531 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0017040 con 0x55f86e3b4ad0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f1896ffd640 1 -- 172.21.15.99:0/2260806531 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f18b000fd60 con 0x55f86e3b4ad0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f86e3a1630 con 0x55f86e3b4ad0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f1896ffd640 1 -- 172.21.15.99:0/2260806531 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f18b0041050 con 0x55f86e3b4ad0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 10 client.16530 fetch_fsmap learned FSMap version 38 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f86e42f440 con 0x55f86e3b4ad0 2024-05-14T17:13:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f1896ffd640 1 -- 172.21.15.99:0/2260806531 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f86e3a1630 con 0x55f86e3b4ad0 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f1896ffd640 1 -- 172.21.15.99:0/2260806531 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f18b0026440 con 0x55f86e3b4ad0 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 10 client.16530 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 20 client.16530 populate_metadata read hostname 'smithi099' 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e42fac0 0x55f86e431ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 4 client.16530 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f86e318360 con 0x55f86e42fac0 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.700+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e42fac0 0x55f86e431ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.701+0000 7f18b77fe640 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e42fac0 0x55f86e431ee0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0188289d1e5d03f server_cookie=143647b49e626e0f in_seq=0 out_seq=0 2024-05-14T17:13:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.701+0000 7f1896ffd640 10 client.16530 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f1896ffd640 1 -- 172.21.15.99:0/2260806531 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f86e318360 con 0x55f86e42fac0 2024-05-14T17:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f1896ffd640 10 client.16530 handle_command_reply: tid=0 2024-05-14T17:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 client.16530 shutdown 2024-05-14T17:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 msgr2=0x55f86e3af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 0x55f86e3af0c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f18b0011460 tx=0x7f18b0003a40 comp rx=0 tx=0).stop 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 shutdown_connections 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55f86e42fac0 0x55f86e431ee0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f86e3b4ad0 0x55f86e3af0c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f86e3aa210 0x55f86e3aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 --2- 172.21.15.99:0/2260806531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f86e3a98f0 0x55f86e3ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 >> 172.21.15.99:0/2260806531 conn(0x55f86e3a5e00 msgr2=0x55f86e3a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f1896ffd640 0 client.16530 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 shutdown_connections 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 1 -- 172.21.15.99:0/2260806531 wait complete. 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.752+0000 7f18c09d6740 20 client.16530 trim_cache size 0 max 16384 2024-05-14T17:13:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:13:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f18a40b2a10 msgr2=0x7f18a40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f18a40b2a10 0x7f18a40b4ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f18b0022000 tx=0x7f18b0023000 comp rx=0 tx=0).stop 2024-05-14T17:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 msgr2=0x7f18b81991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81991f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f18a8005940 tx=0x7f18a80040c0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 shutdown_connections 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f18a40b2a10 0x7f18a40b4ed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18b810d0d0 0x7f18b819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b8104620 0x7f18b8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 --2- 172.21.15.99:0/2747226654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18b8103d00 0x7f18b81991f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 >> 172.21.15.99:0/2747226654 conn(0x7f18b8077cd0 msgr2=0x7f18b80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 shutdown_connections 2024-05-14T17:13:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:13:55.753+0000 7f18bdbe5640 1 -- 172.21.15.99:0/2747226654 wait complete. 2024-05-14T17:13:56.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 4.43 MB/sec execute 368 sec latency 242.490 ms 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: pgmap v322: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 872 KiB/s wr, 83 op/s 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:57 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: pgmap v322: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 872 KiB/s wr, 83 op/s 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:57 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: pgmap v322: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 872 KiB/s wr, 83 op/s 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:57 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:57.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 4.42 MB/sec execute 369 sec latency 1242.581 ms 2024-05-14T17:13:58.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 4.41 MB/sec execute 370 sec latency 2242.608 ms 2024-05-14T17:13:59.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 441547 4.39 MB/sec execute 371 sec latency 3242.681 ms 2024-05-14T17:13:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: pgmap v323: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 872 KiB/s wr, 83 op/s 2024-05-14T17:13:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:59.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:59.705 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:13:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: pgmap v323: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 872 KiB/s wr, 83 op/s 2024-05-14T17:13:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.765 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:59.766 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:59.766 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:13:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: pgmap v323: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 872 KiB/s wr, 83 op/s 2024-05-14T17:13:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:13:59.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:59.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:59.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:13:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: Adjusting osd_memory_target on smithi099 to 176.1M 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: Unable to set osd_memory_target on smithi099 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-14T17:14:00.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:00 smithi099 ceph-mon[31851]: pgmap v324: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-14T17:14:00.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 443867 4.42 MB/sec execute 372 sec latency 3256.956 ms 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: Adjusting osd_memory_target on smithi099 to 176.1M 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: Unable to set osd_memory_target on smithi099 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-14T17:14:00.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:00 smithi150 ceph-mon[36204]: pgmap v324: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-14T17:14:00.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T17:14:00.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T17:14:00.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-05-14T17:14:00.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: Unable to set osd_memory_target on smithi150 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-14T17:14:00.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: Adjusting osd_memory_target on smithi099 to 176.1M 2024-05-14T17:14:00.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: Unable to set osd_memory_target on smithi099 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-14T17:14:00.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:00 smithi193 ceph-mon[35962]: pgmap v324: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-14T17:14:01.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 4.42 MB/sec execute 373 sec latency 289.237 ms 2024-05-14T17:14:02.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 4.41 MB/sec execute 374 sec latency 1289.317 ms 2024-05-14T17:14:03.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:02 smithi150 ceph-mon[36204]: pgmap v325: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 650 KiB/s wr, 110 op/s 2024-05-14T17:14:03.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:02 smithi099 ceph-mon[31851]: pgmap v325: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 650 KiB/s wr, 110 op/s 2024-05-14T17:14:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:02 smithi193 ceph-mon[35962]: pgmap v325: 97 pgs: 97 active+clean; 114 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 650 KiB/s wr, 110 op/s 2024-05-14T17:14:03.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 4.40 MB/sec execute 375 sec latency 2289.391 ms 2024-05-14T17:14:04.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 445096 4.39 MB/sec execute 376 sec latency 3289.490 ms 2024-05-14T17:14:05.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:04 smithi150 ceph-mon[36204]: pgmap v326: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1006 KiB/s wr, 168 op/s 2024-05-14T17:14:05.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:04 smithi099 ceph-mon[31851]: pgmap v326: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1006 KiB/s wr, 168 op/s 2024-05-14T17:14:05.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:04 smithi193 ceph-mon[35962]: pgmap v326: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1006 KiB/s wr, 168 op/s 2024-05-14T17:14:05.550 INFO:tasks.workunit.client.0.smithi099.stdout: 1 446754 4.40 MB/sec execute 377 sec latency 3338.868 ms 2024-05-14T17:14:06.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448197 4.41 MB/sec execute 378 sec latency 19.347 ms 2024-05-14T17:14:07.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:06 smithi150 ceph-mon[36204]: pgmap v327: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 684 KiB/s wr, 127 op/s 2024-05-14T17:14:07.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:06 smithi150 ceph-mon[36204]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:14:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:06 smithi099 ceph-mon[31851]: pgmap v327: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 684 KiB/s wr, 127 op/s 2024-05-14T17:14:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:06 smithi099 ceph-mon[31851]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:14:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:06 smithi193 ceph-mon[35962]: pgmap v327: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 684 KiB/s wr, 127 op/s 2024-05-14T17:14:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:06 smithi193 ceph-mon[35962]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:14:07.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 4.40 MB/sec execute 379 sec latency 766.907 ms 2024-05-14T17:14:08.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 4.39 MB/sec execute 380 sec latency 1767.011 ms 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.853+0000 7f1c89138640 1 -- 172.21.15.99:0/1619259692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c84106400 msgr2=0x7f1c8410dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.853+0000 7f1c89138640 1 --2- 172.21.15.99:0/1619259692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c84106400 0x7f1c8410dfd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009ca0 tx=0x7f1c7c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 -- 172.21.15.99:0/1619259692 shutdown_connections 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 --2- 172.21.15.99:0/1619259692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c84106400 0x7f1c8410dfd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 --2- 172.21.15.99:0/1619259692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101ce0 0x7f1c84105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 --2- 172.21.15.99:0/1619259692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c84101330 0x7f1c84101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 -- 172.21.15.99:0/1619259692 >> 172.21.15.99:0/1619259692 conn(0x7f1c84077e70 msgr2=0x7f1c840ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 -- 172.21.15.99:0/1619259692 shutdown_connections 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 -- 172.21.15.99:0/1619259692 wait complete. 2024-05-14T17:14:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 Processor -- start 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.854+0000 7f1c89138640 1 -- start start 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c89138640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101330 0x7f1c84199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c89138640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c84106400 0x7f1c8419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c89138640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 0x7f1c841a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c89138640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84111350 con 0x7f1c8419aaa0 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c89138640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841111d0 con 0x7f1c84101330 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c89138640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841114d0 con 0x7f1c84106400 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 0x7f1c841a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 0x7f1c841a5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42002/0 (socket says 172.21.15.99:42002) 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 -- 172.21.15.99:0/3774113049 learned_addr learned my addr 172.21.15.99:0/3774113049 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c84106400 0x7f1c8419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101330 0x7f1c84199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 -- 172.21.15.99:0/3774113049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c84106400 msgr2=0x7f1c8419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c84106400 0x7f1c8419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 -- 172.21.15.99:0/3774113049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101330 msgr2=0x7f1c84199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101330 0x7f1c84199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 -- 172.21.15.99:0/3774113049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c841a5820 con 0x7f1c8419aaa0 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101330 0x7f1c84199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c8359e640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 0x7f1c841a5050 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c005cc0 tx=0x7f1c7c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.855+0000 7f1c80d99640 1 -- 172.21.15.99:0/3774113049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c003bf0 con 0x7f1c8419aaa0 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.856+0000 7f1c80d99640 1 -- 172.21.15.99:0/3774113049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c003d90 con 0x7f1c8419aaa0 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.856+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c841a5ab0 con 0x7f1c8419aaa0 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.856+0000 7f1c80d99640 1 -- 172.21.15.99:0/3774113049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c02fbc0 con 0x7f1c8419aaa0 2024-05-14T17:14:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.856+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841a6000 con 0x7f1c8419aaa0 2024-05-14T17:14:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.858+0000 7f1c80d99640 1 -- 172.21.15.99:0/3774113049 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1c7c02fd60 con 0x7f1c8419aaa0 2024-05-14T17:14:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.858+0000 7f1c8be54740 1 Processor -- start 2024-05-14T17:14:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.858+0000 7f1c8be54740 1 -- start start 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe635590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe637ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c8be54740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe59dc30 con 0x55bbbe635ad0 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c8be54740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe58d7d0 con 0x55bbbe62b210 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c8be54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe591810 con 0x55bbbe62a8f0 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe637ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe637ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42010/0 (socket says 172.21.15.99:42010) 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c7bfff640 1 -- 172.21.15.99:0/3283175251 learned_addr learned my addr 172.21.15.99:0/3283175251 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe635590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.859+0000 7f1c80d99640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1c700b2a10 0x7f1c700b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c7bfff640 1 -- 172.21.15.99:0/3283175251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 msgr2=0x55bbbe62acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c7bfff640 1 -- 172.21.15.99:0/3283175251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 msgr2=0x55bbbe635590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe635590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c7bfff640 1 -- 172.21.15.99:0/3283175251 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbbe2a2210 con 0x55bbbe635ad0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c8359e640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1c700b2a10 0x7f1c700b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c80d99640 1 -- 172.21.15.99:0/3774113049 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f1c7c103d20 con 0x7f1c8419aaa0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe637ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c013910 tx=0x7f1c6c00e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525b54a08ab5f2f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c797fa640 1 -- 172.21.15.99:0/3283175251 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004020 con 0x55bbbe635ad0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c797fa640 1 -- 172.21.15.99:0/3283175251 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c00bdc0 con 0x55bbbe635ad0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.860+0000 7f1c797fa640 1 -- 172.21.15.99:0/3283175251 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c003850 con 0x55bbbe635ad0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 -- 172.21.15.99:0/3283175251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 msgr2=0x55bbbe637ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8359e640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1c700b2a10 0x7f1c700b4ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c101600 tx=0x7f1c7c097a10 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe637ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c013910 tx=0x7f1c6c00e430 comp rx=0 tx=0).stop 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 -- 172.21.15.99:0/3283175251 shutdown_connections 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe637ec0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe635590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 --2- 172.21.15.99:0/3283175251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 -- 172.21.15.99:0/3283175251 >> 172.21.15.99:0/3283175251 conn(0x55bbbe626e00 msgr2=0x55bbbe6271e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 -- 172.21.15.99:0/3283175251 shutdown_connections 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 -- 172.21.15.99:0/3283175251 wait complete. 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 -- start start 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.861+0000 7f1c8be54740 1 Processor -- start 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe6353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 0x55bbbe6326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe635ad0 0x55bbbe632c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe273790 con 0x55bbbe62b210 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe549890 con 0x55bbbe62a8f0 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bbbe6387a0 con 0x55bbbe635ad0 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 10 client.0 _ll_register_callbacks cb 0x55bbbe54cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 0x55bbbe6326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 0x55bbbe6326e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42016/0 (socket says 172.21.15.99:42016) 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 -- 172.21.15.99:0/92246326 learned_addr learned my addr 172.21.15.99:0/92246326 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe6353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe635ad0 0x55bbbe632c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 -- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe635ad0 msgr2=0x55bbbe632c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe635ad0 0x55bbbe632c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 -- 172.21.15.99:0/92246326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 msgr2=0x55bbbe6353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe6353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 -- 172.21.15.99:0/92246326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbbe36e620 con 0x55bbbe62b210 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 0x55bbbe6326e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009f10 tx=0x7f1c740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c651a008987ff7df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/92246326 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74012920 con 0x55bbbe62b210 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/92246326 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f1c74012ac0 con 0x55bbbe62b210 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.862+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6c00bdc0 con 0x55bbbe62b210 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/92246326 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1c74012cc0 con 0x55bbbe62b210 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 10 client.16542 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bbbe644210 con 0x55bbbe62b210 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/92246326 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c00bdc0 con 0x55bbbe62b210 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/92246326 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f1c7400f040 con 0x55bbbe62b210 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 10 client.16542 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 20 client.16542 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe645790 0x55bbbe647c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 4 client.16542 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbbe59fb80 con 0x55bbbe645790 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.863+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe645790 0x55bbbe647c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.864+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe645790 0x55bbbe647c20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b143e37e7b0c4116 server_cookie=4bd91b1805a07156 in_seq=0 out_seq=0 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.864+0000 7f1c6a7fc640 10 client.16542 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.864+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/92246326 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbbe59fb80 con 0x55bbbe645790 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.864+0000 7f1c6a7fc640 10 client.16542 handle_command_reply: tid=0 2024-05-14T17:14:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.864+0000 7f1c8be54740 1 client.16542 shutdown 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 msgr2=0x55bbbe6326e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 0x55bbbe6326e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009f10 tx=0x7f1c740117b0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 shutdown_connections 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe645790 0x55bbbe647c20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe635ad0 0x55bbbe632c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe62b210 0x55bbbe6326e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c6a7fc640 0 client.16542 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 --2- 172.21.15.99:0/92246326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe6353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 >> 172.21.15.99:0/92246326 conn(0x55bbbe626e00 msgr2=0x55bbbe626100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 shutdown_connections 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 1 -- 172.21.15.99:0/92246326 wait complete. 2024-05-14T17:14:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.865+0000 7f1c8be54740 20 client.16542 trim_cache size 0 max 16384 2024-05-14T17:14:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 Processor -- start 2024-05-14T17:14:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 -- start start 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62b210 0x55bbbe641670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe634f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841b0f80 con 0x55bbbe635ad0 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ae790 con 0x55bbbe62a8f0 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c8be54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ac600 con 0x55bbbe62b210 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe634f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe634f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42018/0 (socket says 172.21.15.99:42018) 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c7bfff640 1 -- 172.21.15.99:0/263996076 learned_addr learned my addr 172.21.15.99:0/263996076 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.880+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62b210 0x55bbbe641670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c7bfff640 1 -- 172.21.15.99:0/263996076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62b210 msgr2=0x55bbbe641670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62b210 0x55bbbe641670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c7bfff640 1 -- 172.21.15.99:0/263996076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 msgr2=0x55bbbe62d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c7bfff640 1 -- 172.21.15.99:0/263996076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbbe622830 con 0x55bbbe635ad0 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe634f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c004570 tx=0x7f1c6c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea636df4a2298f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c78ff9640 1 -- 172.21.15.99:0/263996076 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c013110 con 0x55bbbe635ad0 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c78ff9640 1 -- 172.21.15.99:0/263996076 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c0132b0 con 0x55bbbe635ad0 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c78ff9640 1 -- 172.21.15.99:0/263996076 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c02d8c0 con 0x55bbbe635ad0 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c8be54740 1 -- 172.21.15.99:0/263996076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 msgr2=0x55bbbe634f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.881+0000 7f1c8be54740 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe634f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c004570 tx=0x7f1c6c0044c0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- 172.21.15.99:0/263996076 shutdown_connections 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe634f20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62b210 0x55bbbe641670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 --2- 172.21.15.99:0/263996076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62a8f0 0x55bbbe62d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- 172.21.15.99:0/263996076 >> 172.21.15.99:0/263996076 conn(0x55bbbe626e00 msgr2=0x55bbbe631730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- 172.21.15.99:0/263996076 shutdown_connections 2024-05-14T17:14:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- 172.21.15.99:0/263996076 wait complete. 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- start start 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 Processor -- start 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe63dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe62ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe62f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841acc50 con 0x55bbbe635ad0 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ae0c0 con 0x55bbbe62b210 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841ad320 con 0x55bbbe62a8f0 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c8be54740 10 client.0 _ll_register_callbacks cb 0x7f1c74009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe62f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe62f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42020/0 (socket says 172.21.15.99:42020) 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c7bfff640 1 -- 172.21.15.99:0/2713896590 learned_addr learned my addr 172.21.15.99:0/2713896590 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe63dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.882+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe62ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c7bfff640 1 -- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 msgr2=0x55bbbe63dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe63dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c7bfff640 1 -- 172.21.15.99:0/2713896590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 msgr2=0x55bbbe62ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe62ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c7bfff640 1 -- 172.21.15.99:0/2713896590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbbe62fa10 con 0x55bbbe635ad0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c7bfff640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe62f4d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00ee60 tx=0x7f1c6c0136d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f3bb7d9167d022 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/2713896590 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00b910 con 0x55bbbe635ad0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/2713896590 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8565+0+0 (secure 0 0 0) 0x7f1c6c00bab0 con 0x55bbbe635ad0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6c0132b0 con 0x55bbbe635ad0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/2713896590 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c046050 con 0x55bbbe635ad0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c8be54740 10 client.16548 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bbbe6afd20 con 0x55bbbe635ad0 2024-05-14T17:14:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/2713896590 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c0132b0 con 0x55bbbe635ad0 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.883+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/2713896590 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f1c6c03d040 con 0x55bbbe635ad0 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c8be54740 10 client.16548 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c8be54740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c8be54740 20 client.16548 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c8be54740 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe6b01a0 0x55bbbe6b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c8be54740 4 client.16548 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbbe599360 con 0x55bbbe6b01a0 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe6b01a0 0x55bbbe6b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c82d9d640 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe6b01a0 0x55bbbe6b25c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ce0354cd44e36c server_cookie=c2231c28959234d5 in_seq=0 out_seq=0 2024-05-14T17:14:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.884+0000 7f1c6a7fc640 10 client.16548 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.931+0000 7f1c6a7fc640 1 -- 172.21.15.99:0/2713896590 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bbbe599360 con 0x55bbbe6b01a0 2024-05-14T17:14:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.931+0000 7f1c6a7fc640 10 client.16548 handle_command_reply: tid=0 2024-05-14T17:14:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.931+0000 7f1c8be54740 1 client.16548 shutdown 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.931+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 msgr2=0x55bbbe62f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.931+0000 7f1c8be54740 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe62f4d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00ee60 tx=0x7f1c6c0136d0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 shutdown_connections 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bbbe6b01a0 0x55bbbe6b25c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbbe635ad0 0x55bbbe62f4d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbbe62b210 0x55bbbe62ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 --2- 172.21.15.99:0/2713896590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bbbe62a8f0 0x55bbbe63dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 >> 172.21.15.99:0/2713896590 conn(0x55bbbe626e00 msgr2=0x55bbbe625870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c6a7fc640 0 client.16548 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 shutdown_connections 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 1 -- 172.21.15.99:0/2713896590 wait complete. 2024-05-14T17:14:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.932+0000 7f1c8be54740 20 client.16548 trim_cache size 0 max 16384 2024-05-14T17:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1c700b2a10 msgr2=0x7f1c700b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1c700b2a10 0x7f1c700b4ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c101600 tx=0x7f1c7c097a10 comp rx=0 tx=0).stop 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 msgr2=0x7f1c841a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 0x7f1c841a5050 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c005cc0 tx=0x7f1c7c02f380 comp rx=0 tx=0).stop 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 shutdown_connections 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f1c700b2a10 0x7f1c700b4ed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c8419aaa0 0x7f1c841a5050 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c84106400 0x7f1c8419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 --2- 172.21.15.99:0/3774113049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c84101330 0x7f1c84199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 >> 172.21.15.99:0/3774113049 conn(0x7f1c84077e70 msgr2=0x7f1c840ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 shutdown_connections 2024-05-14T17:14:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:08.933+0000 7f1c89138640 1 -- 172.21.15.99:0/3774113049 wait complete. 2024-05-14T17:14:09.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:08 smithi150 ceph-mon[36204]: pgmap v329: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 821 KiB/s wr, 153 op/s 2024-05-14T17:14:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:08 smithi099 ceph-mon[31851]: pgmap v329: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 821 KiB/s wr, 153 op/s 2024-05-14T17:14:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:08 smithi193 ceph-mon[35962]: pgmap v329: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 821 KiB/s wr, 153 op/s 2024-05-14T17:14:09.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 448638 4.38 MB/sec execute 381 sec latency 2767.101 ms 2024-05-14T17:14:10.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 450877 4.40 MB/sec execute 382 sec latency 2816.321 ms 2024-05-14T17:14:11.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:10 smithi150 ceph-mon[36204]: pgmap v330: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 929 KiB/s wr, 139 op/s 2024-05-14T17:14:11.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:10 smithi150 ceph-mon[36204]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:14:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:10 smithi099 ceph-mon[31851]: pgmap v330: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 929 KiB/s wr, 139 op/s 2024-05-14T17:14:11.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:10 smithi099 ceph-mon[31851]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:14:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:10 smithi193 ceph-mon[35962]: pgmap v330: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 929 KiB/s wr, 139 op/s 2024-05-14T17:14:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:10 smithi193 ceph-mon[35962]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:14:11.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 452187 4.41 MB/sec execute 383 sec latency 208.624 ms 2024-05-14T17:14:12.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 452187 4.39 MB/sec execute 384 sec latency 1208.661 ms 2024-05-14T17:14:13.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:12 smithi150 ceph-mon[36204]: pgmap v332: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 628 KiB/s wr, 87 op/s 2024-05-14T17:14:13.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:12 smithi099 ceph-mon[31851]: pgmap v332: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 628 KiB/s wr, 87 op/s 2024-05-14T17:14:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:12 smithi193 ceph-mon[35962]: pgmap v332: 97 pgs: 97 active+clean; 117 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 628 KiB/s wr, 87 op/s 2024-05-14T17:14:13.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 452187 4.38 MB/sec execute 385 sec latency 2208.712 ms 2024-05-14T17:14:14.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 452187 4.37 MB/sec execute 386 sec latency 3208.815 ms 2024-05-14T17:14:14.787 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.9 2024-05-14T17:14:14.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:14:15.050 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:14:15.173 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:14 smithi099 ceph-mon[31851]: pgmap v333: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-14T17:14:15.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:14 smithi150 ceph-mon[36204]: pgmap v333: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-14T17:14:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:14 smithi193 ceph-mon[35962]: pgmap v333: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-05-14T17:14:15.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 454412 4.40 MB/sec execute 387 sec latency 3227.990 ms 2024-05-14T17:14:16.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455730 4.40 MB/sec execute 388 sec latency 97.125 ms 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 -- 172.21.15.99:0/2705631354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc075770 msgr2=0x7fe0bc075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/2705631354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc075770 0x7fe0bc075b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009990 tx=0x7fe0ac009ca0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 -- 172.21.15.99:0/2705631354 shutdown_connections 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/2705631354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc114710 0x7fe0bc116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/2705631354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc075770 0x7fe0bc075b50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/2705631354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc076090 0x7fe0bc1141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 -- 172.21.15.99:0/2705631354 >> 172.21.15.99:0/2705631354 conn(0x7fe0bc07a6a0 msgr2=0x7fe0bc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:16.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 -- 172.21.15.99:0/2705631354 shutdown_connections 2024-05-14T17:14:16.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.791+0000 7fe0c10b8700 1 -- 172.21.15.99:0/2705631354 wait complete. 2024-05-14T17:14:16.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 Processor -- start 2024-05-14T17:14:16.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 -- start start 2024-05-14T17:14:16.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc075770 0x7fe0bc1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:16.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 0x7fe0bc1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:16.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc114710 0x7fe0bc1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:16.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc119d90 con 0x7fe0bc076090 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc119c10 con 0x7fe0bc075770 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.792+0000 7fe0c10b8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc119f10 con 0x7fe0bc114710 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 0x7fe0bc1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 0x7fe0bc1a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56254/0 (socket says 172.21.15.99:56254) 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 -- 172.21.15.99:0/3631216078 learned_addr learned my addr 172.21.15.99:0/3631216078 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0bb59e700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc114710 0x7fe0bc1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0bad9d700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc075770 0x7fe0bc1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 -- 172.21.15.99:0/3631216078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc114710 msgr2=0x7fe0bc1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc114710 0x7fe0bc1a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 -- 172.21.15.99:0/3631216078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc075770 msgr2=0x7fe0bc1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc075770 0x7fe0bc1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 -- 172.21.15.99:0/3631216078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc1a52b0 con 0x7fe0bc076090 2024-05-14T17:14:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0bad9d700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc075770 0x7fe0bc1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b3fff700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 0x7fe0bc1a07e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400d980 tx=0x7fe0a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4017070 con 0x7fe0bc076090 2024-05-14T17:14:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a4005020 con 0x7fe0bc076090 2024-05-14T17:14:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc1a55a0 con 0x7fe0bc076090 2024-05-14T17:14:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4009b50 con 0x7fe0bc076090 2024-05-14T17:14:16.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.793+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc1ace20 con 0x7fe0bc076090 2024-05-14T17:14:16.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.794+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc06a230 con 0x7fe0bc076090 2024-05-14T17:14:16.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.796+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe0a40051d0 con 0x7fe0bc076090 2024-05-14T17:14:16.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.796+0000 7fe0b8d99700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0a80b29f0 0x7fe0a80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:16.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.796+0000 7fe0bad9d700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0a80b29f0 0x7fe0a80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:16.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.796+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fe0a40e2950 con 0x7fe0bc076090 2024-05-14T17:14:16.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.797+0000 7fe0bad9d700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0a80b29f0 0x7fe0a80b4eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006010 tx=0x7fe0ac00b540 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:16.800+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0a4020020 con 0x7fe0bc076090 2024-05-14T17:14:17.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.056+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0bc1ad020 con 0x7fe0bc076090 2024-05-14T17:14:17.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.057+0000 7fe0b8d99700 1 -- 172.21.15.99:0/3631216078 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11879 (secure 0 0 0) 0x7fe0a40aa330 con 0x7fe0bc076090 2024-05-14T17:14:17.057 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:14:17.057 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:13:10.003659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0a80b29f0 msgr2=0x7fe0a80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0a80b29f0 0x7fe0a80b4eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006010 tx=0x7fe0ac00b540 comp rx=0 tx=0).stop 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 msgr2=0x7fe0bc1a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 0x7fe0bc1a07e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400d980 tx=0x7fe0a400dc90 comp rx=0 tx=0).stop 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 shutdown_connections 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe0a80b29f0 0x7fe0a80b4eb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc114710 0x7fe0bc1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0bc075770 0x7fe0bc1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 --2- 172.21.15.99:0/3631216078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0bc076090 0x7fe0bc1a07e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.059+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 >> 172.21.15.99:0/3631216078 conn(0x7fe0bc07a6a0 msgr2=0x7fe0bc113540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:17.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.060+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 shutdown_connections 2024-05-14T17:14:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:17.060+0000 7fe0c10b8700 1 -- 172.21.15.99:0/3631216078 wait complete. 2024-05-14T17:14:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 38 2024-05-14T17:14:17.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:16 smithi150 ceph-mon[36204]: pgmap v334: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-14T17:14:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:16 smithi099 ceph-mon[31851]: pgmap v334: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-14T17:14:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:16 smithi193 ceph-mon[35962]: pgmap v334: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-05-14T17:14:17.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455730 4.39 MB/sec execute 389 sec latency 1097.188 ms 2024-05-14T17:14:17.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 137.9 --await-for 45.0' 2024-05-14T17:14:17.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 137.9 --await-for 45.0 2024-05-14T17:14:17.963 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:14:18.198 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:17 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3631216078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:14:18.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:17 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3631216078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:14:18.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:17 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3631216078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:14:18.551 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455730 4.38 MB/sec execute 390 sec latency 2097.213 ms 2024-05-14T17:14:19.078 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:18 smithi099 ceph-mon[31851]: pgmap v335: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-14T17:14:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:18 smithi150 ceph-mon[36204]: pgmap v335: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-14T17:14:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:18 smithi193 ceph-mon[35962]: pgmap v335: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-05-14T17:14:19.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455730 4.36 MB/sec execute 391 sec latency 3097.241 ms 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 -- 172.21.15.99:0/3447610699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 msgr2=0x7f886406b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 --2- 172.21.15.99:0/3447610699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f886406b860 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f88540099c0 tx=0x7f8854009cd0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 -- 172.21.15.99:0/3447610699 shutdown_connections 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 --2- 172.21.15.99:0/3447610699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8864108900 0x7f8864110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 --2- 172.21.15.99:0/3447610699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f886406b860 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 --2- 172.21.15.99:0/3447610699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 0x7f88641081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 -- 172.21.15.99:0/3447610699 >> 172.21.15.99:0/3447610699 conn(0x7f88640793e0 msgr2=0x7f88640797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 -- 172.21.15.99:0/3447610699 shutdown_connections 2024-05-14T17:14:19.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.599+0000 7f886a652700 1 -- 172.21.15.99:0/3447610699 wait complete. 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 Processor -- start 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 -- start start 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f8864102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 0x7f8864102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8864108900 0x7f8864102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8864113730 con 0x7f886406b480 2024-05-14T17:14:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88641135b0 con 0x7f8864108900 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.600+0000 7f886a652700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88641138b0 con 0x7f886406be30 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f8864102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f8864102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56270/0 (socket says 172.21.15.99:56270) 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 -- 172.21.15.99:0/1475018228 learned_addr learned my addr 172.21.15.99:0/1475018228 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f885bfff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 0x7f8864102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8868bef700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8864108900 0x7f8864102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 -- 172.21.15.99:0/1475018228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 msgr2=0x7f8864102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 0x7f8864102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 -- 172.21.15.99:0/1475018228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8864108900 msgr2=0x7f8864102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8864108900 0x7f8864102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 -- 172.21.15.99:0/1475018228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88641a8070 con 0x7f886406b480 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f885bfff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 0x7f8864102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8863fff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f8864102510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8854006010 tx=0x7f88540055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8861ffb700 1 -- 172.21.15.99:0/1475018228 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854026070 con 0x7f886406b480 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8861ffb700 1 -- 172.21.15.99:0/1475018228 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f885400f460 con 0x7f886406b480 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88641a8300 con 0x7f886406b480 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f8861ffb700 1 -- 172.21.15.99:0/1475018228 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854009e60 con 0x7f886406b480 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.601+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88641a8750 con 0x7f886406b480 2024-05-14T17:14:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.604+0000 7f8861ffb700 1 -- 172.21.15.99:0/1475018228 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8854029070 con 0x7f886406b480 2024-05-14T17:14:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.604+0000 7f8861ffb700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88500b2aa0 0x7f88500b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8861ffb700 1 -- 172.21.15.99:0/1475018228 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f88540e2e30 con 0x7f886406b480 2024-05-14T17:14:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f885bfff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88500b2aa0 0x7f88500b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f885bfff700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88500b2aa0 0x7f88500b4f60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f88641082a0 tx=0x7f884c00b5f0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 Processor -- start 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 -- start start 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a0592210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a058c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a058cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600a04eae20 con 0x5600a0591e30 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600a04da9c0 con 0x5600a058cb60 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.605+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5600a04dea00 con 0x5600a0592750 2024-05-14T17:14:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a0592210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a0592210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56284/0 (socket says 172.21.15.99:56284) 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 -- 172.21.15.99:0/15239542 learned_addr learned my addr 172.21.15.99:0/15239542 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8868bef700 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a058cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f885bfff700 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a058c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 -- 172.21.15.99:0/15239542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 msgr2=0x5600a058c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a058c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 -- 172.21.15.99:0/15239542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 msgr2=0x5600a058cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a058cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 -- 172.21.15.99:0/15239542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a042aa60 con 0x5600a0591e30 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f8863fff700 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a0592210 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f88540e0b80 tx=0x7f88540a9990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8e37cefdabcf39 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f885a7fc700 1 -- 172.21.15.99:0/15239542 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540aa1a0 con 0x5600a0591e30 2024-05-14T17:14:19.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f885a7fc700 1 -- 172.21.15.99:0/15239542 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88540ae550 con 0x5600a0591e30 2024-05-14T17:14:19.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.606+0000 7f885a7fc700 1 -- 172.21.15.99:0/15239542 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540ae320 con 0x5600a0591e30 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 -- 172.21.15.99:0/15239542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 msgr2=0x5600a0592210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a0592210 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f88540e0b80 tx=0x7f88540a9990 comp rx=0 tx=0).stop 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 -- 172.21.15.99:0/15239542 shutdown_connections 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a058cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a0592210 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/15239542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a058c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 -- 172.21.15.99:0/15239542 >> 172.21.15.99:0/15239542 conn(0x5600a04b73e0 msgr2=0x5600a0586d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 -- 172.21.15.99:0/15239542 shutdown_connections 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.607+0000 7f8874ee8b80 1 -- 172.21.15.99:0/15239542 wait complete. 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 -- start start 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 Processor -- start 2024-05-14T17:14:19.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a0598410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0592750 0x5600a0595cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600a03e1130 con 0x5600a0591e30 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600a047c950 con 0x5600a0592750 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5600a0598f50 con 0x5600a058cb60 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8874ee8b80 10 client.0 _ll_register_callbacks cb 0x5600a0471710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f885bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f885bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56300/0 (socket says 172.21.15.99:56300) 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f885bfff700 1 -- 172.21.15.99:0/520908338 learned_addr learned my addr 172.21.15.99:0/520908338 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.608+0000 7f8863fff700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a0598410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f8868bef700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0592750 0x5600a0595cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f885bfff700 1 -- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 msgr2=0x5600a0598410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f885bfff700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a0598410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f885bfff700 1 -- 172.21.15.99:0/520908338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0592750 msgr2=0x5600a0595cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f885bfff700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0592750 0x5600a0595cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f885bfff700 1 -- 172.21.15.99:0/520908338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a0479820 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f8868bef700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0592750 0x5600a0595cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f885bfff700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05957b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f884c00fe00 tx=0x7f884c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dae5963cc6336e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f88437fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f88437fe700 1 -- 172.21.15.99:0/520908338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c01f630 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f88437fe700 1 -- 172.21.15.99:0/520908338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f884c01f7d0 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88540ae550 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f88437fe700 1 -- 172.21.15.99:0/520908338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f884c01f9d0 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f8874ee8b80 10 client.16566 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5600a05a3600 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f88437fe700 1 -- 172.21.15.99:0/520908338 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88540ae550 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.609+0000 7f88437fe700 1 -- 172.21.15.99:0/520908338 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f884c0309b0 con 0x5600a0591e30 2024-05-14T17:14:19.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8874ee8b80 10 client.16566 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8874ee8b80 10 client.16566 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8874ee8b80 20 client.16566 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a05a5c30 0x5600a05a8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8874ee8b80 4 client.16566 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600a05584f0 con 0x5600a05a5c30 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8863fff700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a05a5c30 0x5600a05a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f8863fff700 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a05a5c30 0x5600a05a8050 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8fcef1b879c208 server_cookie=511c6455798e1b03 in_seq=0 out_seq=0 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.610+0000 7f88437fe700 10 client.16566 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f88437fe700 1 -- 172.21.15.99:0/520908338 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600a05584f0 con 0x5600a05a5c30 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f88437fe700 10 client.16566 handle_command_reply: tid=0 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 client.16566 shutdown 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 msgr2=0x5600a05957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05957b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f884c00fe00 tx=0x7f884c004750 comp rx=0 tx=0).stop 2024-05-14T17:14:19.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 shutdown_connections 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0592750 0x5600a0595cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a0598410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a05a5c30 0x5600a05a8050 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/520908338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05957b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 >> 172.21.15.99:0/520908338 conn(0x5600a04b73e0 msgr2=0x5600a0581860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f88437fe700 0 client.16566 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 shutdown_connections 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.611+0000 7f8874ee8b80 1 -- 172.21.15.99:0/520908338 wait complete. 2024-05-14T17:14:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.612+0000 7f8874ee8b80 20 client.16566 trim_cache size 0 max 16384 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.632+0000 7f8874ee8b80 1 Processor -- start 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.632+0000 7f8874ee8b80 1 -- start start 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a05a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a0581970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88641b27f0 con 0x5600a0591e30 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88641affc0 con 0x5600a058cb60 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88641ade30 con 0x5600a0592750 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56314/0 (socket says 172.21.15.99:56314) 2024-05-14T17:14:19.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 -- 172.21.15.99:0/1601378737 learned_addr learned my addr 172.21.15.99:0/1601378737 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8868bef700 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a0581970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8863fff700 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a05a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 -- 172.21.15.99:0/1601378737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 msgr2=0x5600a0581970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a0581970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 -- 172.21.15.99:0/1601378737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 msgr2=0x5600a05a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a05a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 -- 172.21.15.99:0/1601378737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a0590050 con 0x5600a0591e30 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f885bfff700 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05933c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f884c014040 tx=0x7f884c00fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e77d6c4c18c9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.633+0000 7f8859ffb700 1 -- 172.21.15.99:0/1601378737 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c036040 con 0x5600a0591e30 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8859ffb700 1 -- 172.21.15.99:0/1601378737 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c01fb70 con 0x5600a0591e30 2024-05-14T17:14:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8859ffb700 1 -- 172.21.15.99:0/1601378737 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c0052a0 con 0x5600a0591e30 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 -- 172.21.15.99:0/1601378737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 msgr2=0x5600a05933c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05933c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f884c014040 tx=0x7f884c00fef0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 -- 172.21.15.99:0/1601378737 shutdown_connections 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a0592750 0x5600a0581970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a058cb60 0x5600a05a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/1601378737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0591e30 0x5600a05933c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 -- 172.21.15.99:0/1601378737 >> 172.21.15.99:0/1601378737 conn(0x5600a04b73e0 msgr2=0x5600a05940a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 -- 172.21.15.99:0/1601378737 shutdown_connections 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.634+0000 7f8874ee8b80 1 -- 172.21.15.99:0/1601378737 wait complete. 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 -- start start 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 Processor -- start 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a059e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0591e30 0x5600a059e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 0x5600a059b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88641ae480 con 0x5600a0592750 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88641af8f0 con 0x5600a0591e30 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88641aeb50 con 0x5600a058cb60 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8874ee8b80 10 client.0 _ll_register_callbacks cb 0x5600a0471710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8868bef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 0x5600a059b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f885bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0591e30 0x5600a059e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8863fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a059e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8868bef700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 0x5600a059b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56318/0 (socket says 172.21.15.99:56318) 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f8868bef700 1 -- 172.21.15.99:0/4281069749 learned_addr learned my addr 172.21.15.99:0/4281069749 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.635+0000 7f885bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0591e30 0x5600a059e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:58934/0 (socket says 172.21.15.99:58934) 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8868bef700 1 -- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 msgr2=0x5600a059e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8868bef700 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a059e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8868bef700 1 -- 172.21.15.99:0/4281069749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0591e30 msgr2=0x5600a059e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8868bef700 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0591e30 0x5600a059e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8868bef700 1 -- 172.21.15.99:0/4281069749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a059b6e0 con 0x5600a0592750 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8868bef700 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 0x5600a059b170 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f885c007fa0 tx=0x7f885c0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd879dd9af25417 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f88437fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f88437fe700 1 -- 172.21.15.99:0/4281069749 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c018d30 con 0x5600a0592750 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f88437fe700 1 -- 172.21.15.99:0/4281069749 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f885c0203f0 con 0x5600a0592750 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c01fb70 con 0x5600a0592750 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f88437fe700 1 -- 172.21.15.99:0/4281069749 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f885c0205f0 con 0x5600a0592750 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8874ee8b80 10 client.16572 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5600a0610d60 con 0x5600a0592750 2024-05-14T17:14:19.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f88437fe700 1 -- 172.21.15.99:0/4281069749 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c01fb70 con 0x5600a0592750 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.636+0000 7f88437fe700 1 -- 172.21.15.99:0/4281069749 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f885c01f030 con 0x5600a0592750 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8874ee8b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8874ee8b80 10 client.16572 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8874ee8b80 20 client.16572 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a06115e0 0x5600a0613a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8874ee8b80 4 client.16572 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.9, "await_for": 45.0}] 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.9, "await_for": 45.0}) -- 0x5600a04ecd70 con 0x5600a06115e0 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8863fff700 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a06115e0 0x5600a0613a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f8863fff700 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a06115e0 0x5600a0613a00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb1b4da5aea3832a server_cookie=dc520785a9933138 in_seq=0 out_seq=0 2024-05-14T17:14:19.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.637+0000 7f88437fe700 10 client.16572 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.697+0000 7f88437fe700 1 -- 172.21.15.99:0/4281069749 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5600a04ecd70 con 0x5600a06115e0 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.697+0000 7f88437fe700 10 client.16572 handle_command_reply: tid=0 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.697+0000 7f8874ee8b80 1 client.16572 shutdown 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 msgr2=0x5600a059b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 0x5600a059b170 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f885c007fa0 tx=0x7f885c0094e0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 shutdown_connections 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5600a06115e0 0x5600a0613a00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600a0592750 0x5600a059b170 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f88437fe700 0 client.16572 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600a058cb60 0x5600a059e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 --2- 172.21.15.99:0/4281069749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600a0591e30 0x5600a059e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 >> 172.21.15.99:0/4281069749 conn(0x5600a04b73e0 msgr2=0x5600a0593240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 shutdown_connections 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 1 -- 172.21.15.99:0/4281069749 wait complete. 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.698+0000 7f8874ee8b80 20 client.16572 trim_cache size 0 max 16384 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 38, 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 34, 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "d6cbbda9": { 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 34, 2024-05-14T17:14:19.699 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 137.9, 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:19.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88500b2aa0 msgr2=0x7f88500b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88500b2aa0 0x7f88500b4f60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f88641082a0 tx=0x7f884c00b5f0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 msgr2=0x7f8864102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f8864102510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8854006010 tx=0x7f88540055f0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 shutdown_connections 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88500b2aa0 0x7f88500b4f60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8864108900 0x7f8864102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f886406b480 0x7f8864102510 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 --2- 172.21.15.99:0/1475018228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886406be30 0x7f8864102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 >> 172.21.15.99:0/1475018228 conn(0x7f88640793e0 msgr2=0x7f88641003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 shutdown_connections 2024-05-14T17:14:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:19.699+0000 7f886a652700 1 -- 172.21.15.99:0/1475018228 wait complete. 2024-05-14T17:14:20.356 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd6cbbda9', quiesce took 5.6 seconds. Will release after: 12.299999999999999 2024-05-14T17:14:20.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.35 MB/sec execute 392 sec latency 3150.130 ms 2024-05-14T17:14:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:20 smithi150 ceph-mon[36204]: pgmap v336: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-14T17:14:21.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:20 smithi099 ceph-mon[31851]: pgmap v336: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-14T17:14:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:20 smithi193 ceph-mon[35962]: pgmap v336: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-05-14T17:14:21.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.34 MB/sec execute 393 sec latency 1885.681 ms 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 -- 172.21.15.99:0/3001604601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741045c0 msgr2=0x7f897410caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 --2- 172.21.15.99:0/3001604601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741045c0 0x7f897410caa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8968009ca0 tx=0x7f896802a950 comp rx=0 tx=0).stop 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 -- 172.21.15.99:0/3001604601 shutdown_connections 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 --2- 172.21.15.99:0/3001604601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f897410d070 0x7f897410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 --2- 172.21.15.99:0/3001604601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741045c0 0x7f897410caa0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 --2- 172.21.15.99:0/3001604601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974103ca0 0x7f8974104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 -- 172.21.15.99:0/3001604601 >> 172.21.15.99:0/3001604601 conn(0x7f8974078b60 msgr2=0x7f8974078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 -- 172.21.15.99:0/3001604601 shutdown_connections 2024-05-14T17:14:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.031+0000 7f8978a69640 1 -- 172.21.15.99:0/3001604601 wait complete. 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 Processor -- start 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 -- start start 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974103ca0 0x7f89741991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89741045c0 0x7f8974199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 0x7f897419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8974112d40 con 0x7f897410d070 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8974112bc0 con 0x7f89741045c0 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8978a69640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8974112ec0 con 0x7f8974103ca0 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8972d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 0x7f897419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8972d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 0x7f897419daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56334/0 (socket says 172.21.15.99:56334) 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8972d9d640 1 -- 172.21.15.99:0/706609331 learned_addr learned my addr 172.21.15.99:0/706609331 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.032+0000 7f8971d9b640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89741045c0 0x7f8974199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f897259c640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974103ca0 0x7f89741991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8972d9d640 1 -- 172.21.15.99:0/706609331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974103ca0 msgr2=0x7f89741991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8972d9d640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974103ca0 0x7f89741991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8972d9d640 1 -- 172.21.15.99:0/706609331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89741045c0 msgr2=0x7f8974199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8972d9d640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89741045c0 0x7f8974199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8972d9d640 1 -- 172.21.15.99:0/706609331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897419e1d0 con 0x7f897410d070 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8972d9d640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 0x7f897419daa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c720 tx=0x7f896c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f89637fe640 1 -- 172.21.15.99:0/706609331 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c007bb0 con 0x7f897410d070 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897419e4c0 con 0x7f897410d070 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f89637fe640 1 -- 172.21.15.99:0/706609331 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896c00ce30 con 0x7f897410d070 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f89637fe640 1 -- 172.21.15.99:0/706609331 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c004db0 con 0x7f897410d070 2024-05-14T17:14:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.033+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89741a5d50 con 0x7f897410d070 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.035+0000 7f89637fe640 1 -- 172.21.15.99:0/706609331 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f896c005020 con 0x7f897410d070 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 Processor -- start 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 -- start start 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95eee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95efb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564b95e5cc30 con 0x564b95eedd00 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564b95e4c7d0 con 0x564b95ef8ee0 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897b778740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b95e50810 con 0x564b95eee620 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95eee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95eee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56348/0 (socket says 172.21.15.99:56348) 2024-05-14T17:14:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f8971d9b640 1 -- 172.21.15.99:0/2759039383 learned_addr learned my addr 172.21.15.99:0/2759039383 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f897259c640 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.036+0000 7f89637fe640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f895c0b2a10 0x7f895c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95efb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 -- 172.21.15.99:0/2759039383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 msgr2=0x564b95ef89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8972d9d640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f895c0b2a10 0x7f895c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 -- 172.21.15.99:0/2759039383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 msgr2=0x564b95efb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95efb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 -- 172.21.15.99:0/2759039383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b95b61210 con 0x564b95eedd00 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f89637fe640 1 -- 172.21.15.99:0/706609331 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f896c0e20d0 con 0x7f897410d070 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8971d9b640 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95eee0e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8968009a60 tx=0x7f8968005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962575e9122cb5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8960ff9640 1 -- 172.21.15.99:0/2759039383 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968034040 con 0x564b95eedd00 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8972d9d640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f895c0b2a10 0x7f895c0b4ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f896c0df970 tx=0x7f896c091040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8960ff9640 1 -- 172.21.15.99:0/2759039383 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896803e030 con 0x564b95eedd00 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.037+0000 7f8960ff9640 1 -- 172.21.15.99:0/2759039383 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968030c30 con 0x564b95eedd00 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 -- 172.21.15.99:0/2759039383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 msgr2=0x564b95eee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95eee0e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8968009a60 tx=0x7f8968005fa0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 -- 172.21.15.99:0/2759039383 shutdown_connections 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95efb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 --2- 172.21.15.99:0/2759039383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95eee0e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 -- 172.21.15.99:0/2759039383 >> 172.21.15.99:0/2759039383 conn(0x564b95eea210 msgr2=0x564b95eea5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 -- 172.21.15.99:0/2759039383 shutdown_connections 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 -- 172.21.15.99:0/2759039383 wait complete. 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.038+0000 7f897b778740 1 -- start start 2024-05-14T17:14:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 Processor -- start 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95eedd00 0x564b95ef8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 0x564b95ef6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564b95b32790 con 0x564b95ef8ee0 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564b95e08890 con 0x564b95eedd00 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b95efbb50 con 0x564b95eee620 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897b778740 10 client.0 _ll_register_callbacks cb 0x564b95e0bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 0x564b95ef6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 0x564b95ef6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56364/0 (socket says 172.21.15.99:56364) 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 -- 172.21.15.99:0/3594210570 learned_addr learned my addr 172.21.15.99:0/3594210570 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f897259c640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95eedd00 0x564b95ef8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 -- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 msgr2=0x564b95ef5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 -- 172.21.15.99:0/3594210570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95eedd00 msgr2=0x564b95ef8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95eedd00 0x564b95ef8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 -- 172.21.15.99:0/3594210570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b95c2d620 con 0x564b95ef8ee0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.039+0000 7f8971d9b640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 0x564b95ef6110 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f896802a990 tx=0x7f89680308d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcca71d6aadcfe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f8959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f8959ffb640 1 -- 172.21.15.99:0/3594210570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968056060 con 0x564b95ef8ee0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f8959ffb640 1 -- 172.21.15.99:0/3594210570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f896802f870 con 0x564b95ef8ee0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896803e030 con 0x564b95ef8ee0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f8959ffb640 1 -- 172.21.15.99:0/3594210570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8968034040 con 0x564b95ef8ee0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 10 client.16584 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564b95f075f0 con 0x564b95ef8ee0 2024-05-14T17:14:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f8959ffb640 1 -- 172.21.15.99:0/3594210570 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896803e030 con 0x564b95ef8ee0 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f8959ffb640 1 -- 172.21.15.99:0/3594210570 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f896803a790 con 0x564b95ef8ee0 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 10 client.16584 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 20 client.16584 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f08940 0x564b95f0ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 4 client.16584 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.040+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b95e5eb80 con 0x564b95f08940 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.041+0000 7f897259c640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f08940 0x564b95f0ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.041+0000 7f897259c640 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f08940 0x564b95f0ad30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb46f740530d3d9e server_cookie=89707542c685453e in_seq=0 out_seq=0 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.041+0000 7f8959ffb640 10 client.16584 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f8959ffb640 1 -- 172.21.15.99:0/3594210570 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b95e5eb80 con 0x564b95f08940 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f8959ffb640 10 client.16584 handle_command_reply: tid=0 2024-05-14T17:14:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 client.16584 shutdown 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 msgr2=0x564b95ef6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 0x564b95ef6110 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f896802a990 tx=0x7f89680308d0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 shutdown_connections 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f08940 0x564b95f0ad30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95ef8ee0 0x564b95ef6110 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ef5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f8959ffb640 0 client.16584 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 --2- 172.21.15.99:0/3594210570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95eedd00 0x564b95ef8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 >> 172.21.15.99:0/3594210570 conn(0x564b95eea210 msgr2=0x564b95eef700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 shutdown_connections 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 1 -- 172.21.15.99:0/3594210570 wait complete. 2024-05-14T17:14:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.042+0000 7f897b778740 20 client.16584 trim_cache size 0 max 16384 2024-05-14T17:14:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 Processor -- start 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 -- start start 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95ee5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ee0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95ef0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89741aed70 con 0x564b95eedd00 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89741ac580 con 0x564b95ef8ee0 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897b778740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89741aa410 con 0x564b95eee620 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95ee5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95ee5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56378/0 (socket says 172.21.15.99:56378) 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f8971d9b640 1 -- 172.21.15.99:0/1485970376 learned_addr learned my addr 172.21.15.99:0/1485970376 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.057+0000 7f897259c640 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ee0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95ef0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 -- 172.21.15.99:0/1485970376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 msgr2=0x564b95ee0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ee0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 -- 172.21.15.99:0/1485970376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 msgr2=0x564b95ef0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95ef0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 -- 172.21.15.99:0/1485970376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b95ef4b20 con 0x564b95eedd00 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f8971d9b640 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95ee5100 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f896803abe0 tx=0x7f8968009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942458a36bdfad43 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f895bfff640 1 -- 172.21.15.99:0/1485970376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896804d040 con 0x564b95eedd00 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f895bfff640 1 -- 172.21.15.99:0/1485970376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896803e070 con 0x564b95eedd00 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f895bfff640 1 -- 172.21.15.99:0/1485970376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968056040 con 0x564b95eedd00 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f897b778740 1 -- 172.21.15.99:0/1485970376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 msgr2=0x564b95ee5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.058+0000 7f897b778740 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95ee5100 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f896803abe0 tx=0x7f8968009800 comp rx=0 tx=0).stop 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- 172.21.15.99:0/1485970376 shutdown_connections 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95ef0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eee620 0x564b95ee0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 --2- 172.21.15.99:0/1485970376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eedd00 0x564b95ee5100 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- 172.21.15.99:0/1485970376 >> 172.21.15.99:0/1485970376 conn(0x564b95eea210 msgr2=0x564b95ef0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- 172.21.15.99:0/1485970376 shutdown_connections 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- 172.21.15.99:0/1485970376 wait complete. 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- start start 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 Processor -- start 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eedd00 0x564b95f0b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 0x564b95f0bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95f0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89741aaa60 con 0x564b95eee620 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89741abeb0 con 0x564b95ef8ee0 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89741ab110 con 0x564b95eedd00 2024-05-14T17:14:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897b778740 10 client.0 _ll_register_callbacks cb 0x7f8968030e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 0x564b95f0bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 0x564b95f0bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56394/0 (socket says 172.21.15.99:56394) 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 -- 172.21.15.99:0/849705252 learned_addr learned my addr 172.21.15.99:0/849705252 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f8971d9b640 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eedd00 0x564b95f0b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 -- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eedd00 msgr2=0x564b95f0b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eedd00 0x564b95f0b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 -- 172.21.15.99:0/849705252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 msgr2=0x564b95f0c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.059+0000 7f897259c640 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95f0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897259c640 1 -- 172.21.15.99:0/849705252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b95f71ee0 con 0x564b95eee620 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897259c640 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 0x564b95f0bef0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8964004700 tx=0x7f8964010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363d3f0d84f8dd56 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f8959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f8959ffb640 1 -- 172.21.15.99:0/849705252 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964004220 con 0x564b95eee620 2024-05-14T17:14:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f8959ffb640 1 -- 172.21.15.99:0/849705252 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f89640043c0 con 0x564b95eee620 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896803e070 con 0x564b95eee620 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f8959ffb640 1 -- 172.21.15.99:0/849705252 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f896401c560 con 0x564b95eee620 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 10 client.16590 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564b95f73260 con 0x564b95eee620 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f8959ffb640 1 -- 172.21.15.99:0/849705252 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896803e070 con 0x564b95eee620 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f8959ffb640 1 -- 172.21.15.99:0/849705252 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f8964025480 con 0x564b95eee620 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 10 client.16590 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 20 client.16590 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f73a70 0x564b95f75e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 4 client.16590 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.060+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b95e58360 con 0x564b95f73a70 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.061+0000 7f8971d9b640 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f73a70 0x564b95f75e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.061+0000 7f8959ffb640 10 client.16590 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.061+0000 7f8971d9b640 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f73a70 0x564b95f75e90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db0300c5d8601df server_cookie=4c0200a87eebbec5 in_seq=0 out_seq=0 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.097+0000 7f8959ffb640 1 -- 172.21.15.99:0/849705252 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564b95e58360 con 0x564b95f73a70 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f8959ffb640 10 client.16590 handle_command_reply: tid=0 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 client.16590 shutdown 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 msgr2=0x564b95f0bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 0x564b95f0bef0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8964004700 tx=0x7f8964010af0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 shutdown_connections 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564b95f73a70 0x564b95f75e90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b95ef8ee0 0x564b95f0c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b95eee620 0x564b95f0bef0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f8959ffb640 0 client.16590 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 --2- 172.21.15.99:0/849705252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b95eedd00 0x564b95f0b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 >> 172.21.15.99:0/849705252 conn(0x564b95eea210 msgr2=0x564b95ee4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 shutdown_connections 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 1 -- 172.21.15.99:0/849705252 wait complete. 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.098+0000 7f897b778740 20 client.16590 trim_cache size 0 max 16384 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:14:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f895c0b2a10 msgr2=0x7f895c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f895c0b2a10 0x7f895c0b4ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f896c0df970 tx=0x7f896c091040 comp rx=0 tx=0).stop 2024-05-14T17:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 msgr2=0x7f897419daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 0x7f897419daa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c720 tx=0x7f896c00cbe0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 shutdown_connections 2024-05-14T17:14:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f895c0b2a10 0x7f895c0b4ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f897410d070 0x7f897419daa0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89741045c0 0x7f8974199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 --2- 172.21.15.99:0/706609331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8974103ca0 0x7f89741991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 >> 172.21.15.99:0/706609331 conn(0x7f8974078b60 msgr2=0x7f89740fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 shutdown_connections 2024-05-14T17:14:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:22.099+0000 7f8978a69640 1 -- 172.21.15.99:0/706609331 wait complete. 2024-05-14T17:14:22.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.33 MB/sec execute 394 sec latency 2885.772 ms 2024-05-14T17:14:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:22 smithi150 ceph-mon[36204]: pgmap v337: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-05-14T17:14:23.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:22 smithi099 ceph-mon[31851]: pgmap v337: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-05-14T17:14:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:22 smithi193 ceph-mon[35962]: pgmap v337: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-05-14T17:14:23.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.32 MB/sec execute 395 sec latency 3885.878 ms 2024-05-14T17:14:24.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.31 MB/sec execute 396 sec latency 4885.947 ms 2024-05-14T17:14:25.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:24 smithi150 ceph-mon[36204]: pgmap v338: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:14:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:24 smithi099 ceph-mon[31851]: pgmap v338: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:14:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:24 smithi193 ceph-mon[35962]: pgmap v338: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-05-14T17:14:25.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.30 MB/sec execute 397 sec latency 5886.000 ms 2024-05-14T17:14:26.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.29 MB/sec execute 398 sec latency 6886.091 ms 2024-05-14T17:14:27.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:26 smithi150 ceph-mon[36204]: pgmap v339: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 880 KiB/s wr, 78 op/s 2024-05-14T17:14:27.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:26 smithi099 ceph-mon[31851]: pgmap v339: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 880 KiB/s wr, 78 op/s 2024-05-14T17:14:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:26 smithi193 ceph-mon[35962]: pgmap v339: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 880 KiB/s wr, 78 op/s 2024-05-14T17:14:27.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.28 MB/sec execute 399 sec latency 7886.243 ms 2024-05-14T17:14:28.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.27 MB/sec execute 400 sec latency 8886.367 ms 2024-05-14T17:14:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:28 smithi150 ceph-mon[36204]: pgmap v340: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 880 KiB/s wr, 78 op/s 2024-05-14T17:14:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:28 smithi099 ceph-mon[31851]: pgmap v340: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 880 KiB/s wr, 78 op/s 2024-05-14T17:14:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:28 smithi193 ceph-mon[35962]: pgmap v340: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 880 KiB/s wr, 78 op/s 2024-05-14T17:14:29.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.26 MB/sec execute 401 sec latency 9886.460 ms 2024-05-14T17:14:30.552 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.25 MB/sec execute 402 sec latency 10886.510 ms 2024-05-14T17:14:31.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:30 smithi150 ceph-mon[36204]: pgmap v341: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-05-14T17:14:31.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:30 smithi099 ceph-mon[31851]: pgmap v341: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-05-14T17:14:31.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:30 smithi193 ceph-mon[35962]: pgmap v341: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-05-14T17:14:31.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.24 MB/sec execute 403 sec latency 11886.611 ms 2024-05-14T17:14:32.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.22 MB/sec execute 404 sec latency 12886.733 ms 2024-05-14T17:14:32.657 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:14:32.912 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:14:33.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:32 smithi150 ceph-mon[36204]: pgmap v342: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 655 KiB/s wr, 45 op/s 2024-05-14T17:14:33.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:32 smithi099 ceph-mon[31851]: pgmap v342: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 655 KiB/s wr, 45 op/s 2024-05-14T17:14:33.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:32 smithi193 ceph-mon[35962]: pgmap v342: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 655 KiB/s wr, 45 op/s 2024-05-14T17:14:33.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.21 MB/sec execute 405 sec latency 13886.808 ms 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 -- 172.21.15.99:0/4067153044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3141095b0 msgr2=0x7fe314109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/4067153044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3141095b0 0x7fe314109990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe308009990 tx=0x7fe308009ca0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 -- 172.21.15.99:0/4067153044 shutdown_connections 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/4067153044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 0x7fe3141103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/4067153044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3141095b0 0x7fe314109990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/4067153044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31406b220 0x7fe31406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 -- 172.21.15.99:0/4067153044 >> 172.21.15.99:0/4067153044 conn(0x7fe3140793f0 msgr2=0x7fe314079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 -- 172.21.15.99:0/4067153044 shutdown_connections 2024-05-14T17:14:34.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.541+0000 7fe31ab0d700 1 -- 172.21.15.99:0/4067153044 wait complete. 2024-05-14T17:14:34.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 Processor -- start 2024-05-14T17:14:34.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 -- start start 2024-05-14T17:14:34.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 0x7fe31419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 0x7fe31419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141095b0 0x7fe3141a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141136c0 con 0x7fe31406b220 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe314113540 con 0x7fe31406bbe0 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe31ab0d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe314113840 con 0x7fe3141095b0 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 0x7fe31419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 0x7fe31419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53066/0 (socket says 172.21.15.99:53066) 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 -- 172.21.15.99:0/2016257151 learned_addr learned my addr 172.21.15.99:0/2016257151 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe313fff700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 0x7fe31419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 -- 172.21.15.99:0/2016257151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141095b0 msgr2=0x7fe3141a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141095b0 0x7fe3141a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 -- 172.21.15.99:0/2016257151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 msgr2=0x7fe31419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 0x7fe31419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe3188a9700 1 -- 172.21.15.99:0/2016257151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3141a0d60 con 0x7fe31406b220 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.542+0000 7fe313fff700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 0x7fe31419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:34.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.543+0000 7fe3188a9700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 0x7fe31419bd50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3080057c0 tx=0x7fe3080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:34.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.543+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308027070 con 0x7fe31406b220 2024-05-14T17:14:34.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.543+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe308004020 con 0x7fe31406b220 2024-05-14T17:14:34.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.543+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3141a0ff0 con 0x7fe31406b220 2024-05-14T17:14:34.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.543+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308022bf0 con 0x7fe31406b220 2024-05-14T17:14:34.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.543+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3141a88d0 con 0x7fe31406b220 2024-05-14T17:14:34.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.544+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31406a230 con 0x7fe31406b220 2024-05-14T17:14:34.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.545+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe30802a070 con 0x7fe31406b220 2024-05-14T17:14:34.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.546+0000 7fe311ffb700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe3040b29f0 0x7fe3040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.546+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fe3080e3d40 con 0x7fe31406b220 2024-05-14T17:14:34.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.546+0000 7fe313fff700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe3040b29f0 0x7fe3040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:34.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.546+0000 7fe313fff700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe3040b29f0 0x7fe3040b4eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe300005fd0 tx=0x7fe300005e30 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:34.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.549+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe308031020 con 0x7fe31406b220 2024-05-14T17:14:34.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.20 MB/sec execute 406 sec latency 14886.879 ms 2024-05-14T17:14:34.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.811+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3141a8aa0 con 0x7fe31406b220 2024-05-14T17:14:34.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.811+0000 7fe311ffb700 1 -- 172.21.15.99:0/2016257151 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11879 (secure 0 0 0) 0x7fe308013070 con 0x7fe31406b220 2024-05-14T17:14:34.811 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:14:34.811 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:13:10.003659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:14:34.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe3040b29f0 msgr2=0x7fe3040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe3040b29f0 0x7fe3040b4eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe300005fd0 tx=0x7fe300005e30 comp rx=0 tx=0).stop 2024-05-14T17:14:34.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 msgr2=0x7fe31419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:34.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 0x7fe31419bd50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3080057c0 tx=0x7fe3080057f0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 shutdown_connections 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe3040b29f0 0x7fe3040b4eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3141095b0 0x7fe3141a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31406b220 0x7fe31419bd50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 --2- 172.21.15.99:0/2016257151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31406bbe0 0x7fe31419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 >> 172.21.15.99:0/2016257151 conn(0x7fe3140793f0 msgr2=0x7fe314100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 shutdown_connections 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:34.814+0000 7fe31ab0d700 1 -- 172.21.15.99:0/2016257151 wait complete. 2024-05-14T17:14:34.814 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 38 2024-05-14T17:14:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 -- 172.21.15.99:0/2113917439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5201068f0 msgr2=0x7fa520106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 --2- 172.21.15.99:0/2113917439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5201068f0 0x7fa520106cd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002a410 comp rx=0 tx=0).stop 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 -- 172.21.15.99:0/2113917439 shutdown_connections 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 --2- 172.21.15.99:0/2113917439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102660 0x7fa52010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 --2- 172.21.15.99:0/2113917439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa520101ca0 0x7fa520102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 --2- 172.21.15.99:0/2113917439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5201068f0 0x7fa520106cd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.187+0000 7fa525e23640 1 -- 172.21.15.99:0/2113917439 >> 172.21.15.99:0/2113917439 conn(0x7fa520077750 msgr2=0x7fa520077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.188+0000 7fa525e23640 1 -- 172.21.15.99:0/2113917439 shutdown_connections 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.188+0000 7fa525e23640 1 -- 172.21.15.99:0/2113917439 wait complete. 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.188+0000 7fa525e23640 1 Processor -- start 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.188+0000 7fa525e23640 1 -- start start 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.188+0000 7fa525e23640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa520101ca0 0x7fa520199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.188+0000 7fa525e23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 0x7fa52019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa525e23640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5201068f0 0x7fa5201a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa525e23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111bd0 con 0x7fa520102660 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa525e23640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111a50 con 0x7fa520101ca0 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa525e23640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111d50 con 0x7fa5201068f0 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 0x7fa52019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 0x7fa52019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53082/0 (socket says 172.21.15.99:53082) 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 -- 172.21.15.99:0/2743702329 learned_addr learned my addr 172.21.15.99:0/2743702329 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51ffff640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5201068f0 0x7fa5201a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa520101ca0 0x7fa520199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 -- 172.21.15.99:0/2743702329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5201068f0 msgr2=0x7fa5201a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5201068f0 0x7fa5201a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 -- 172.21.15.99:0/2743702329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa520101ca0 msgr2=0x7fa520199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa520101ca0 0x7fa520199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 -- 172.21.15.99:0/2743702329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201a54d0 con 0x7fa520102660 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51ffff640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5201068f0 0x7fa5201a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51effd640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 0x7fa52019a450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c920 tx=0x7fa51400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51cff9640 1 -- 172.21.15.99:0/2743702329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514007c70 con 0x7fa520102660 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51cff9640 1 -- 172.21.15.99:0/2743702329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa514007e10 con 0x7fa520102660 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201a56a0 con 0x7fa520102660 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa51cff9640 1 -- 172.21.15.99:0/2743702329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514018820 con 0x7fa520102660 2024-05-14T17:14:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.189+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201a5bf0 con 0x7fa520102660 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa51cff9640 1 -- 172.21.15.99:0/2743702329 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa51400f450 con 0x7fa520102660 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 Processor -- start 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 -- start start 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b080eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b080f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b080f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558b08061c30 con 0x558b080ef500 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558b080517d0 con 0x558b080eebe0 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa528c1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558b08055810 con 0x558b080f3390 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b080f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b080f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53092/0 (socket says 172.21.15.99:53092) 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa51f7fe640 1 -- 172.21.15.99:0/4035267247 learned_addr learned my addr 172.21.15.99:0/4035267247 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa51effd640 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b080f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.193+0000 7fa51ffff640 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b080eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51effd640 1 -- 172.21.15.99:0/4035267247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 msgr2=0x558b080eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51effd640 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b080eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51effd640 1 -- 172.21.15.99:0/4035267247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 msgr2=0x558b080f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51effd640 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b080f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51effd640 1 -- 172.21.15.99:0/4035267247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b07d66210 con 0x558b080f3390 2024-05-14T17:14:35.196 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:34 smithi099 ceph-mon[31851]: pgmap v343: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 924 KiB/s wr, 59 op/s 2024-05-14T17:14:35.196 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:34 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2016257151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b080f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51cff9640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa5080b2a10 0x7fa5080b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51ffff640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa5080b2a10 0x7fa5080b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51effd640 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b080f37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5140dff50 tx=0x7fa5140e8cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c9842b5ee48c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.194+0000 7fa51cff9640 1 -- 172.21.15.99:0/2743702329 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fa5140e2800 con 0x7fa520102660 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.195+0000 7fa505ffb640 1 -- 172.21.15.99:0/4035267247 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140e5e60 con 0x558b080f3390 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.195+0000 7fa505ffb640 1 -- 172.21.15.99:0/4035267247 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5140a95f0 con 0x558b080f3390 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.195+0000 7fa505ffb640 1 -- 172.21.15.99:0/4035267247 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140a98b0 con 0x558b080f3390 2024-05-14T17:14:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.195+0000 7fa528c1e740 1 -- 172.21.15.99:0/4035267247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 msgr2=0x558b080f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.195+0000 7fa51ffff640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa5080b2a10 0x7fa5080b4ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c610 tx=0x7fa51800e040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.195+0000 7fa528c1e740 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b080f37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa5140dff50 tx=0x7fa5140e8cd0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- 172.21.15.99:0/4035267247 shutdown_connections 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b080f37d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b080f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 --2- 172.21.15.99:0/4035267247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b080eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- 172.21.15.99:0/4035267247 >> 172.21.15.99:0/4035267247 conn(0x558b080ea8b0 msgr2=0x558b080eac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- 172.21.15.99:0/4035267247 shutdown_connections 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- 172.21.15.99:0/4035267247 wait complete. 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- start start 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 Processor -- start 2024-05-14T17:14:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b0810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b0810a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0810a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558b07d37790 con 0x558b080ef500 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558b0800d890 con 0x558b080eebe0 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.196+0000 7fa528c1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558b0810d710 con 0x558b080f3390 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa528c1e740 10 client.0 _ll_register_callbacks cb 0x558b08010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b0810a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b0810a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53108/0 (socket says 172.21.15.99:53108) 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 -- 172.21.15.99:0/2452129107 learned_addr learned my addr 172.21.15.99:0/2452129107 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51ffff640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0810a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b0810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 -- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 msgr2=0x558b0810a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0810a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 -- 172.21.15.99:0/2452129107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 msgr2=0x558b0810cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b0810cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 -- 172.21.15.99:0/2452129107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b07e32620 con 0x558b080ef500 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b0810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa51effd640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b0810a100 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa5140e0000 tx=0x7fa5140076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864a3ff45032add2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa4feffd640 1 -- 172.21.15.99:0/2452129107 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140a9250 con 0x558b080ef500 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5140a95f0 con 0x558b080ef500 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa4feffd640 1 -- 172.21.15.99:0/2452129107 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fa51400f740 con 0x558b080ef500 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa4feffd640 1 -- 172.21.15.99:0/2452129107 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa51406e050 con 0x558b080ef500 2024-05-14T17:14:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa528c1e740 10 client.16608 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.197+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558b08119210 con 0x558b080ef500 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa4feffd640 1 -- 172.21.15.99:0/2452129107 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5140a95f0 con 0x558b080ef500 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa4feffd640 1 -- 172.21.15.99:0/2452129107 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fa5140e01d0 con 0x558b080ef500 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa528c1e740 10 client.16608 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa528c1e740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa528c1e740 20 client.16608 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa528c1e740 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b0811a560 0x558b0811c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa528c1e740 4 client.16608 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b08063b80 con 0x558b0811a560 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b0811a560 0x558b0811c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b0811a560 0x558b0811c950 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a1426bc59cdc3f server_cookie=29753085f3b5468f in_seq=0 out_seq=0 2024-05-14T17:14:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.198+0000 7fa4feffd640 10 client.16608 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.199+0000 7fa4feffd640 1 -- 172.21.15.99:0/2452129107 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b08063b80 con 0x558b0811a560 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.199+0000 7fa4feffd640 10 client.16608 handle_command_reply: tid=0 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.199+0000 7fa528c1e740 1 client.16608 shutdown 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 msgr2=0x558b0810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b0810a100 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa5140e0000 tx=0x7fa5140076c0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 shutdown_connections 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b0811a560 0x558b0811c950 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0810a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080ef500 0x558b0810a100 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa4feffd640 0 client.16608 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 --2- 172.21.15.99:0/2452129107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080eebe0 0x558b0810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 >> 172.21.15.99:0/2452129107 conn(0x558b080ea8b0 msgr2=0x558b080f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 shutdown_connections 2024-05-14T17:14:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 1 -- 172.21.15.99:0/2452129107 wait complete. 2024-05-14T17:14:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.200+0000 7fa528c1e740 20 client.16608 trim_cache size 0 max 16384 2024-05-14T17:14:35.208 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:34 smithi150 ceph-mon[36204]: pgmap v343: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 924 KiB/s wr, 59 op/s 2024-05-14T17:14:35.208 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:34 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2016257151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 Processor -- start 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 -- start start 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 0x558b080f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0811feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201aed50 con 0x558b080eebe0 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201ac560 con 0x558b080ef500 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa528c1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201aa3d0 con 0x558b080f3390 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 0x558b080f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 0x558b080f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53114/0 (socket says 172.21.15.99:53114) 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa51f7fe640 1 -- 172.21.15.99:0/1485496360 learned_addr learned my addr 172.21.15.99:0/1485496360 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.215+0000 7fa51effd640 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51f7fe640 1 -- 172.21.15.99:0/1485496360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 msgr2=0x558b0811feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0811feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51f7fe640 1 -- 172.21.15.99:0/1485496360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 msgr2=0x558b0811f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51f7fe640 1 -- 172.21.15.99:0/1485496360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b081203f0 con 0x558b080eebe0 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51effd640 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 0x558b080f7ac0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa510006010 tx=0x7fa510005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a76ceeb72643a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa5057fa640 1 -- 172.21.15.99:0/1485496360 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510035040 con 0x558b080eebe0 2024-05-14T17:14:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa5057fa640 1 -- 172.21.15.99:0/1485496360 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa510005020 con 0x558b080eebe0 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa5057fa640 1 -- 172.21.15.99:0/1485496360 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510004a90 con 0x558b080eebe0 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa528c1e740 1 -- 172.21.15.99:0/1485496360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 msgr2=0x558b080f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa528c1e740 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 0x558b080f7ac0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa510006010 tx=0x7fa510005a20 comp rx=0 tx=0).stop 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa528c1e740 1 -- 172.21.15.99:0/1485496360 shutdown_connections 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa528c1e740 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b080f3390 0x558b0811feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.216+0000 7fa528c1e740 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 --2- 172.21.15.99:0/1485496360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080eebe0 0x558b080f7ac0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- 172.21.15.99:0/1485496360 >> 172.21.15.99:0/1485496360 conn(0x558b080ea8b0 msgr2=0x558b080f73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- 172.21.15.99:0/1485496360 shutdown_connections 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- 172.21.15.99:0/1485496360 wait complete. 2024-05-14T17:14:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- start start 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 Processor -- start 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 0x558b0811c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b0817ca30 0x558b0817ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201aaa20 con 0x558b080f3390 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201abe90 con 0x558b080ef500 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201ab0f0 con 0x558b0817ca30 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa528c1e740 10 client.0 _ll_register_callbacks cb 0x7fa5140e8a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa51effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 0x558b0811c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa51effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 0x558b0811c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53120/0 (socket says 172.21.15.99:53120) 2024-05-14T17:14:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa51effd640 1 -- 172.21.15.99:0/823064419 learned_addr learned my addr 172.21.15.99:0/823064419 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.217+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51ffff640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b0817ca30 0x558b0817ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51effd640 1 -- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b0817ca30 msgr2=0x558b0817ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51effd640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b0817ca30 0x558b0817ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51effd640 1 -- 172.21.15.99:0/823064419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 msgr2=0x558b0811bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51effd640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51effd640 1 -- 172.21.15.99:0/823064419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0817d5a0 con 0x558b080f3390 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa51effd640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 0x558b0811c3e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa5140e0550 tx=0x7fa5140e8cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13af2b5cd3d375ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa4feffd640 1 -- 172.21.15.99:0/823064419 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140e5e60 con 0x558b080f3390 2024-05-14T17:14:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa4feffd640 1 -- 172.21.15.99:0/823064419 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fa51401f300 con 0x558b080f3390 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa510005020 con 0x558b080f3390 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa4feffd640 1 -- 172.21.15.99:0/823064419 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa5140a9430 con 0x558b080f3390 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 10 client.16614 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558b081858b0 con 0x558b080f3390 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa4feffd640 1 -- 172.21.15.99:0/823064419 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa510005020 con 0x558b080f3390 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa4feffd640 1 -- 172.21.15.99:0/823064419 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fa5140b24b0 con 0x558b080f3390 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 10 client.16614 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 20 client.16614 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b08185e70 0x558b08188290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 4 client.16614 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.218+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b0805d360 con 0x558b08185e70 2024-05-14T17:14:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.219+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b08185e70 0x558b08188290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.219+0000 7fa4feffd640 10 client.16614 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.219+0000 7fa51f7fe640 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b08185e70 0x558b08188290 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b500d35bc244c8 server_cookie=90c53856c4d92e31 in_seq=0 out_seq=0 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa4feffd640 1 -- 172.21.15.99:0/823064419 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558b0805d360 con 0x558b08185e70 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa4feffd640 10 client.16614 handle_command_reply: tid=0 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 client.16614 shutdown 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 msgr2=0x558b0811c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 0x558b0811c3e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa5140e0550 tx=0x7fa5140e8cd0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 shutdown_connections 2024-05-14T17:14:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558b08185e70 0x558b08188290 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b0817ca30 0x558b0817ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b080f3390 0x558b0811c3e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa4feffd640 0 client.16614 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 --2- 172.21.15.99:0/823064419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b080ef500 0x558b0811bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 >> 172.21.15.99:0/823064419 conn(0x558b080ea8b0 msgr2=0x558b080f7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 shutdown_connections 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 1 -- 172.21.15.99:0/823064419 wait complete. 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.243+0000 7fa528c1e740 20 client.16614 trim_cache size 0 max 16384 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:14:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa5080b2a10 msgr2=0x7fa5080b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa5080b2a10 0x7fa5080b4ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c610 tx=0x7fa51800e040 comp rx=0 tx=0).stop 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 msgr2=0x7fa52019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 0x7fa52019a450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c920 tx=0x7fa51400cde0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 shutdown_connections 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa5080b2a10 0x7fa5080b4ed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5201068f0 0x7fa5201a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa520102660 0x7fa52019a450 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 --2- 172.21.15.99:0/2743702329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa520101ca0 0x7fa520199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 >> 172.21.15.99:0/2743702329 conn(0x7fa520077750 msgr2=0x7fa5200ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 shutdown_connections 2024-05-14T17:14:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:35.244+0000 7fa525e23640 1 -- 172.21.15.99:0/2743702329 wait complete. 2024-05-14T17:14:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:34 smithi193 ceph-mon[35962]: pgmap v343: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 924 KiB/s wr, 59 op/s 2024-05-14T17:14:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:34 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2016257151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:14:35.495 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id d6cbbda9 --release --await' 2024-05-14T17:14:35.495 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id d6cbbda9 --release --await 2024-05-14T17:14:35.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.19 MB/sec execute 407 sec latency 15886.947 ms 2024-05-14T17:14:35.736 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:14:36.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455813 4.18 MB/sec execute 408 sec latency 16887.016 ms 2024-05-14T17:14:37.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:36 smithi150 ceph-mon[36204]: pgmap v344: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 615 KiB/s wr, 28 op/s 2024-05-14T17:14:37.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:36 smithi099 ceph-mon[31851]: pgmap v344: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 615 KiB/s wr, 28 op/s 2024-05-14T17:14:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:36 smithi193 ceph-mon[35962]: pgmap v344: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 615 KiB/s wr, 28 op/s 2024-05-14T17:14:37.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 -- 172.21.15.99:0/347097637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0060109670 msgr2=0x7f0060109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 --2- 172.21.15.99:0/347097637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0060109670 0x7f0060109a50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f005c0099c0 tx=0x7f005c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 -- 172.21.15.99:0/347097637 shutdown_connections 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 --2- 172.21.15.99:0/347097637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060103fb0 0x7f0060110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 --2- 172.21.15.99:0/347097637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0060109670 0x7f0060109a50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 --2- 172.21.15.99:0/347097637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00601035f0 0x7f0060103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 -- 172.21.15.99:0/347097637 >> 172.21.15.99:0/347097637 conn(0x7f006007a690 msgr2=0x7f006007aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 -- 172.21.15.99:0/347097637 shutdown_connections 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.465+0000 7f00670f5700 1 -- 172.21.15.99:0/347097637 wait complete. 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 Processor -- start 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 -- start start 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 0x7f006019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0060103fb0 0x7f006019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060109670 0x7f0060199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0060113780 con 0x7f00601035f0 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0060113600 con 0x7f0060109670 2024-05-14T17:14:37.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f00670f5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0060113900 con 0x7f0060103fb0 2024-05-14T17:14:37.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 0x7f006019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 0x7f006019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53130/0 (socket says 172.21.15.99:53130) 2024-05-14T17:14:37.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 -- 172.21.15.99:0/606592114 learned_addr learned my addr 172.21.15.99:0/606592114 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0065692700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060109670 0x7f0060199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0057fff700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0060103fb0 0x7f006019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 -- 172.21.15.99:0/606592114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0060103fb0 msgr2=0x7f006019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0060103fb0 0x7f006019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 -- 172.21.15.99:0/606592114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060109670 msgr2=0x7f0060199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060109670 0x7f0060199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0064e91700 1 -- 172.21.15.99:0/606592114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006019a1a0 con 0x7f00601035f0 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0065692700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060109670 0x7f0060199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.466+0000 7f0057fff700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0060103fb0 0x7f006019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:37.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.467+0000 7f0064e91700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 0x7f006019f830 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b580 tx=0x7f005c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:37.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.467+0000 7f0055ffb700 1 -- 172.21.15.99:0/606592114 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c027070 con 0x7f00601035f0 2024-05-14T17:14:37.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.467+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006019a430 con 0x7f00601035f0 2024-05-14T17:14:37.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.467+0000 7f0055ffb700 1 -- 172.21.15.99:0/606592114 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c004020 con 0x7f00601035f0 2024-05-14T17:14:37.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.467+0000 7f0055ffb700 1 -- 172.21.15.99:0/606592114 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c022e30 con 0x7f00601035f0 2024-05-14T17:14:37.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.468+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601a6700 con 0x7f00601035f0 2024-05-14T17:14:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.469+0000 7f0055ffb700 1 -- 172.21.15.99:0/606592114 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f005c022590 con 0x7f00601035f0 2024-05-14T17:14:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.470+0000 7f0055ffb700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00480b29f0 0x7f00480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.470+0000 7f0057fff700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00480b29f0 0x7f00480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.470+0000 7f0055ffb700 1 -- 172.21.15.99:0/606592114 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f005c0e3230 con 0x7f00601035f0 2024-05-14T17:14:37.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.471+0000 7f0057fff700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00480b29f0 0x7f00480b4eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0050005fd0 tx=0x7f0050005dc0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 Processor -- start 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 -- start start 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d88c0 0x5574fd9d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 0x5574fd9e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5574fd932d90 con 0x5574fd9d9280 2024-05-14T17:14:37.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5574fd922930 con 0x5574fd9dd0b0 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f007198bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574fd926970 con 0x5574fd9d88c0 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 0x5574fd9e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 0x5574fd9e4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53146/0 (socket says 172.21.15.99:53146) 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 -- 172.21.15.99:0/3249284241 learned_addr learned my addr 172.21.15.99:0/3249284241 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0064e91700 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0057fff700 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d88c0 0x5574fd9d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 -- 172.21.15.99:0/3249284241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d88c0 msgr2=0x5574fd9d8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d88c0 0x5574fd9d8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 -- 172.21.15.99:0/3249284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 msgr2=0x5574fd9dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9dd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.472+0000 7f0065692700 1 -- 172.21.15.99:0/3249284241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fd8729d0 con 0x5574fd9d9280 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.473+0000 7f0065692700 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 0x5574fd9e4fd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0058009e50 tx=0x7f00580077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d10113c9f9a30b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.473+0000 7f0046ffd700 1 -- 172.21.15.99:0/3249284241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058014610 con 0x5574fd9d9280 2024-05-14T17:14:37.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.473+0000 7f0046ffd700 1 -- 172.21.15.99:0/3249284241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0058019070 con 0x5574fd9d9280 2024-05-14T17:14:37.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.473+0000 7f0046ffd700 1 -- 172.21.15.99:0/3249284241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058013a30 con 0x5574fd9d9280 2024-05-14T17:14:37.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 -- 172.21.15.99:0/3249284241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 msgr2=0x5574fd9e4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 0x5574fd9e4fd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0058009e50 tx=0x7f00580077e0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 -- 172.21.15.99:0/3249284241 shutdown_connections 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d9280 0x5574fd9e4fd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 --2- 172.21.15.99:0/3249284241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d88c0 0x5574fd9d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 -- 172.21.15.99:0/3249284241 >> 172.21.15.99:0/3249284241 conn(0x5574fd8ff350 msgr2=0x5574fd9cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 -- 172.21.15.99:0/3249284241 shutdown_connections 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 -- 172.21.15.99:0/3249284241 wait complete. 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 -- start start 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 Processor -- start 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.474+0000 7f007198bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5574fd8290a0 con 0x5574fd9d88c0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5574fd8c48c0 con 0x5574fd9dd0b0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574fd9e5570 con 0x5574fd9d9280 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 10 client.0 _ll_register_callbacks cb 0x5574fd8b9680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9e4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53160/0 (socket says 172.21.15.99:53160) 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 -- 172.21.15.99:0/1721482398 learned_addr learned my addr 172.21.15.99:0/1721482398 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0057fff700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0065692700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 -- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 msgr2=0x5574fd9e1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 -- 172.21.15.99:0/1721482398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 msgr2=0x5574fd9e2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 -- 172.21.15.99:0/1721482398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fd8c1790 con 0x5574fd9d88c0 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0057fff700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:14:37.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f0064e91700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9e4b80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f005c0e0f80 tx=0x7f005c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af571fe64484ed2d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f003ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f003ffff700 1 -- 172.21.15.99:0/1721482398 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0e3050 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f003ffff700 1 -- 172.21.15.99:0/1721482398 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f005c0b30f0 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0058019070 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f003ffff700 1 -- 172.21.15.99:0/1721482398 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f005c0aaca0 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 10 client.16626 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.475+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5574fd9efb90 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f003ffff700 1 -- 172.21.15.99:0/1721482398 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0058019070 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f003ffff700 1 -- 172.21.15.99:0/1721482398 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f005c0afae0 con 0x5574fd9d88c0 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f007198bb80 10 client.16626 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f007198bb80 10 client.16626 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f007198bb80 20 client.16626 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f007198bb80 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fd9f2100 0x5574fd9f4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f007198bb80 4 client.16626 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574fd9a0460 con 0x5574fd9f2100 2024-05-14T17:14:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.476+0000 7f0057fff700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fd9f2100 0x5574fd9f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f0057fff700 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fd9f2100 0x5574fd9f4550 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23859642a0821e75 server_cookie=5f1eba6d38495452 in_seq=0 out_seq=0 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f003ffff700 10 client.16626 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f003ffff700 1 -- 172.21.15.99:0/1721482398 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574fd9a0460 con 0x5574fd9f2100 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f003ffff700 10 client.16626 handle_command_reply: tid=0 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 client.16626 shutdown 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 msgr2=0x5574fd9e4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9e4b80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f005c0e0f80 tx=0x7f005c0044b0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 shutdown_connections 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fd9f2100 0x5574fd9f4550 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9e4b80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 --2- 172.21.15.99:0/1721482398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 >> 172.21.15.99:0/1721482398 conn(0x5574fd8ff350 msgr2=0x5574fd9de730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.477+0000 7f003ffff700 0 client.16626 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.478+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 shutdown_connections 2024-05-14T17:14:37.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.478+0000 7f007198bb80 1 -- 172.21.15.99:0/1721482398 wait complete. 2024-05-14T17:14:37.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.478+0000 7f007198bb80 20 client.16626 trim_cache size 0 max 16384 2024-05-14T17:14:37.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 Processor -- start 2024-05-14T17:14:37.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 -- start start 2024-05-14T17:14:37.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00601ae3a0 con 0x5574fd9d88c0 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00601abb70 con 0x5574fd9dd0b0 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f007198bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00601a99e0 con 0x5574fd9d9280 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.497+0000 7f0064e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53170/0 (socket says 172.21.15.99:53170) 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 -- 172.21.15.99:0/3612518214 learned_addr learned my addr 172.21.15.99:0/3612518214 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0057fff700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0065692700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 -- 172.21.15.99:0/3612518214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 msgr2=0x5574fd9e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 -- 172.21.15.99:0/3612518214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 msgr2=0x5574fd9e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 -- 172.21.15.99:0/3612518214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fd9d9060 con 0x5574fd9d88c0 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0057fff700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0065692700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f0064e91700 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9ee480 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f005c06b040 tx=0x7f005c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35db2260b425a48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f00467fc700 1 -- 172.21.15.99:0/3612518214 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0e3040 con 0x5574fd9d88c0 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f00467fc700 1 -- 172.21.15.99:0/3612518214 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c0b3370 con 0x5574fd9d88c0 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.498+0000 7f00467fc700 1 -- 172.21.15.99:0/3612518214 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0a9a60 con 0x5574fd9d88c0 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 -- 172.21.15.99:0/3612518214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 msgr2=0x5574fd9ee480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9ee480 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f005c06b040 tx=0x7f005c009cd0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 -- 172.21.15.99:0/3612518214 shutdown_connections 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9d88c0 0x5574fd9ee480 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 --2- 172.21.15.99:0/3612518214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 -- 172.21.15.99:0/3612518214 >> 172.21.15.99:0/3612518214 conn(0x5574fd8ff350 msgr2=0x5574fd9e35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 -- 172.21.15.99:0/3612518214 shutdown_connections 2024-05-14T17:14:37.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 -- 172.21.15.99:0/3612518214 wait complete. 2024-05-14T17:14:37.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 -- start start 2024-05-14T17:14:37.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.499+0000 7f007198bb80 1 Processor -- start 2024-05-14T17:14:37.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9d88c0 0x5574fd9eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00601aa030 con 0x5574fd9dd0b0 2024-05-14T17:14:37.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f00601ab4a0 con 0x5574fd9d88c0 2024-05-14T17:14:37.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00601aa700 con 0x5574fd9d9280 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 10 client.0 _ll_register_callbacks cb 0x5574fd8b9680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53174/0 (socket says 172.21.15.99:53174) 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 -- 172.21.15.99:0/2672656957 learned_addr learned my addr 172.21.15.99:0/2672656957 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0057fff700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0064e91700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9d88c0 0x5574fd9eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 -- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 msgr2=0x5574fd9e77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 -- 172.21.15.99:0/2672656957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9d88c0 msgr2=0x5574fd9eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9d88c0 0x5574fd9eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 -- 172.21.15.99:0/2672656957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fd9e85a0 con 0x5574fd9dd0b0 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0064e91700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9d88c0 0x5574fd9eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f0065692700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e7d30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f00580093e0 tx=0x7f00580060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678aeb390487dbee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:37.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f003ffff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f003ffff700 1 -- 172.21.15.99:0/2672656957 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058013800 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f003ffff700 1 -- 172.21.15.99:0/2672656957 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f00580139a0 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.500+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005c0b3370 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f003ffff700 1 -- 172.21.15.99:0/2672656957 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f00580133e0 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 10 client.16632 fetch_fsmap learned FSMap version 38 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5574fda5da40 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f003ffff700 1 -- 172.21.15.99:0/2672656957 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c0b3370 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f003ffff700 1 -- 172.21.15.99:0/2672656957 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f0058005710 con 0x5574fd9dd0b0 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 10 client.16632 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 10 client.16632 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 20 client.16632 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fda5df20 0x5574fda60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 4 client.16632 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d6cbbda9", "release": true, "await": true}] 2024-05-14T17:14:37.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d6cbbda9", "release": true, "await": true}) -- 0x5574fd934ce0 con 0x5574fda5df20 2024-05-14T17:14:37.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.501+0000 7f0064e91700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fda5df20 0x5574fda60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.502+0000 7f003ffff700 10 client.16632 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:37.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.502+0000 7f0064e91700 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fda5df20 0x5574fda60340 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa937d6de7d55ef6 server_cookie=a3e2fcb386ef9baf in_seq=0 out_seq=0 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f003ffff700 1 -- 172.21.15.99:0/2672656957 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5574fd934ce0 con 0x5574fda5df20 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f003ffff700 10 client.16632 handle_command_reply: tid=0 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 client.16632 shutdown 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 msgr2=0x5574fd9e7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e7d30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f00580093e0 tx=0x7f00580060c0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 shutdown_connections 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5574fda5df20 0x5574fda60340 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574fd9dd0b0 0x5574fd9e7d30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5574fd9d88c0 0x5574fd9eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 --2- 172.21.15.99:0/2672656957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574fd9d9280 0x5574fd9e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 >> 172.21.15.99:0/2672656957 conn(0x5574fd8ff350 msgr2=0x5574fd9d3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f003ffff700 0 client.16632 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 shutdown_connections 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 1 -- 172.21.15.99:0/2672656957 wait complete. 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.506+0000 7f007198bb80 20 client.16632 trim_cache size 0 max 16384 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 38, 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 36, 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:14:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "d6cbbda9": { 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 36, 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 137.9, 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:14:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:14:37.510 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00480b29f0 msgr2=0x7f00480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00480b29f0 0x7f00480b4eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0050005fd0 tx=0x7f0050005dc0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 msgr2=0x7f006019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 0x7f006019f830 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b580 tx=0x7f005c0047b0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 shutdown_connections 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f00480b29f0 0x7f00480b4eb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0060109670 0x7f0060199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00601035f0 0x7f006019f830 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 --2- 172.21.15.99:0/606592114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0060103fb0 0x7f006019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 >> 172.21.15.99:0/606592114 conn(0x7f006007a690 msgr2=0x7f00601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 shutdown_connections 2024-05-14T17:14:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:14:37.507+0000 7f00670f5700 1 -- 172.21.15.99:0/606592114 wait complete. 2024-05-14T17:14:37.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 455864 4.17 MB/sec execute 409 sec latency 17843.793 ms 2024-05-14T17:14:38.230 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd6cbbda9', total seconds elapsed: 23.4 2024-05-14T17:14:38.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 458120 4.20 MB/sec execute 410 sec latency 16.768 ms 2024-05-14T17:14:39.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:38 smithi150 ceph-mon[36204]: pgmap v345: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 615 KiB/s wr, 28 op/s 2024-05-14T17:14:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:38 smithi099 ceph-mon[31851]: pgmap v345: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 615 KiB/s wr, 28 op/s 2024-05-14T17:14:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:38 smithi193 ceph-mon[35962]: pgmap v345: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 615 KiB/s wr, 28 op/s 2024-05-14T17:14:39.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 459246 4.24 MB/sec execute 411 sec latency 38.846 ms 2024-05-14T17:14:40.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 460030 4.24 MB/sec execute 412 sec latency 633.154 ms 2024-05-14T17:14:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:40 smithi099 ceph-mon[31851]: pgmap v346: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-05-14T17:14:41.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:40 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:14:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:40 smithi193 ceph-mon[35962]: pgmap v346: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-05-14T17:14:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:40 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:14:41.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:40 smithi150 ceph-mon[36204]: pgmap v346: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-05-14T17:14:41.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:40 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:14:41.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 460030 4.23 MB/sec execute 413 sec latency 1633.232 ms 2024-05-14T17:14:42.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 460030 4.22 MB/sec execute 414 sec latency 2633.333 ms 2024-05-14T17:14:43.255 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:42 smithi150 ceph-mon[36204]: pgmap v347: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 759 KiB/s wr, 53 op/s 2024-05-14T17:14:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:42 smithi099 ceph-mon[31851]: pgmap v347: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 759 KiB/s wr, 53 op/s 2024-05-14T17:14:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:42 smithi193 ceph-mon[35962]: pgmap v347: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 759 KiB/s wr, 53 op/s 2024-05-14T17:14:43.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 460030 4.21 MB/sec execute 415 sec latency 3633.400 ms 2024-05-14T17:14:44.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 460030 4.20 MB/sec execute 416 sec latency 4633.481 ms 2024-05-14T17:14:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:44 smithi099 ceph-mon[31851]: pgmap v348: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-05-14T17:14:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:44 smithi193 ceph-mon[35962]: pgmap v348: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-05-14T17:14:45.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:44 smithi150 ceph-mon[36204]: pgmap v348: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-05-14T17:14:45.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 462276 4.22 MB/sec execute 417 sec latency 4650.757 ms 2024-05-14T17:14:46.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 4.23 MB/sec execute 418 sec latency 211.047 ms 2024-05-14T17:14:47.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:46 smithi099 ceph-mon[31851]: pgmap v349: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 954 KiB/s wr, 79 op/s 2024-05-14T17:14:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:46 smithi193 ceph-mon[35962]: pgmap v349: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 954 KiB/s wr, 79 op/s 2024-05-14T17:14:47.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:46 smithi150 ceph-mon[36204]: pgmap v349: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 954 KiB/s wr, 79 op/s 2024-05-14T17:14:47.553 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 4.22 MB/sec execute 419 sec latency 1211.131 ms 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 -- 172.21.15.99:0/1622413196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 msgr2=0x7f937410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 --2- 172.21.15.99:0/1622413196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f937410cae0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f935c009900 tx=0x7f935c02a510 comp rx=0 tx=0).stop 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 -- 172.21.15.99:0/1622413196 shutdown_connections 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 --2- 172.21.15.99:0/1622413196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f937410d020 0x7f937410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 --2- 172.21.15.99:0/1622413196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f937410cae0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 --2- 172.21.15.99:0/1622413196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9374103ce0 0x7f93741040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 -- 172.21.15.99:0/1622413196 >> 172.21.15.99:0/1622413196 conn(0x7f9374078ad0 msgr2=0x7f9374078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 -- 172.21.15.99:0/1622413196 shutdown_connections 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.342+0000 7f9379c54640 1 -- 172.21.15.99:0/1622413196 wait complete. 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 Processor -- start 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 -- start start 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9374103ce0 0x7f93741991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f9374199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937410d020 0x7f937419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9374112cd0 con 0x7f9374104600 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9374112b50 con 0x7f937410d020 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9379c54640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9374112e50 con 0x7f9374103ce0 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f936bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f9374199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f936bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f9374199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34052/0 (socket says 172.21.15.99:34052) 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f936bfff640 1 -- 172.21.15.99:0/1925446075 learned_addr learned my addr 172.21.15.99:0/1925446075 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.343+0000 7f9373fff640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937410d020 0x7f937419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f93737fe640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9374103ce0 0x7f93741991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f936bfff640 1 -- 172.21.15.99:0/1925446075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9374103ce0 msgr2=0x7f93741991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f936bfff640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9374103ce0 0x7f93741991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f936bfff640 1 -- 172.21.15.99:0/1925446075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937410d020 msgr2=0x7f937419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f936bfff640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937410d020 0x7f937419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f936bfff640 1 -- 172.21.15.99:0/1925446075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937419e1c0 con 0x7f9374104600 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f93737fe640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9374103ce0 0x7f93741991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f936bfff640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f9374199710 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f935c005940 tx=0x7f935c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f93717fa640 1 -- 172.21.15.99:0/1925446075 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c03f030 con 0x7f9374104600 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f93717fa640 1 -- 172.21.15.99:0/1925446075 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c003cb0 con 0x7f9374104600 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937419e450 con 0x7f9374104600 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f93717fa640 1 -- 172.21.15.99:0/1925446075 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c035070 con 0x7f9374104600 2024-05-14T17:14:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.344+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a5d40 con 0x7f9374104600 2024-05-14T17:14:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.346+0000 7f93717fa640 1 -- 172.21.15.99:0/1925446075 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f935c042030 con 0x7f9374104600 2024-05-14T17:14:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 Processor -- start 2024-05-14T17:14:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 -- start start 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa0acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa15590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa17ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f97dc30 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f96d7d0 con 0x55bf0fa15ad0 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f937c96d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f971810 con 0x55bf0fa0b210 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa0acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa0acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34058/0 (socket says 172.21.15.99:34058) 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 -- 172.21.15.99:0/4289260813 learned_addr learned my addr 172.21.15.99:0/4289260813 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f93737fe640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa15590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f936bfff640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa17ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f93717fa640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f93600b2a10 0x7f93600b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 -- 172.21.15.99:0/4289260813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 msgr2=0x55bf0fa15590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa15590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 -- 172.21.15.99:0/4289260813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 msgr2=0x55bf0fa17ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa17ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f9373fff640 1 -- 172.21.15.99:0/4289260813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0f682210 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f936bfff640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa17ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.347+0000 7f93737fe640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.348+0000 7f93717fa640 1 -- 172.21.15.99:0/1925446075 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f935c102ef0 con 0x7f9374104600 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.348+0000 7f9373fff640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f93600b2a10 0x7f93600b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.348+0000 7f9373fff640 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa0acd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f936c009f70 tx=0x7f936c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d1b7183d51342a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.348+0000 7f9369ffb640 1 -- 172.21.15.99:0/4289260813 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c016070 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.348+0000 7f9369ffb640 1 -- 172.21.15.99:0/4289260813 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c01c440 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.348+0000 7f9369ffb640 1 -- 172.21.15.99:0/4289260813 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c01b5e0 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 -- 172.21.15.99:0/4289260813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 msgr2=0x55bf0fa0acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f9373fff640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f93600b2a10 0x7f93600b4ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f936c011010 tx=0x7f936c01bd40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa0acd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f936c009f70 tx=0x7f936c010800 comp rx=0 tx=0).stop 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 -- 172.21.15.99:0/4289260813 shutdown_connections 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa17ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 --2- 172.21.15.99:0/4289260813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa0acd0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 -- 172.21.15.99:0/4289260813 >> 172.21.15.99:0/4289260813 conn(0x55bf0fa06e00 msgr2=0x55bf0fa071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 -- 172.21.15.99:0/4289260813 shutdown_connections 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 -- 172.21.15.99:0/4289260813 wait complete. 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 -- start start 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.349+0000 7f937c96d740 1 Processor -- start 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa15390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 0x55bf0fa12680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa12bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f653790 con 0x55bf0fa0b210 2024-05-14T17:14:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bf0f929890 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf0fa187a0 con 0x55bf0fa15ad0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 10 client.0 _ll_register_callbacks cb 0x55bf0f92cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 0x55bf0fa12680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 0x55bf0fa12680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34064/0 (socket says 172.21.15.99:34064) 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 -- 172.21.15.99:0/1050287276 learned_addr learned my addr 172.21.15.99:0/1050287276 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f9373fff640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa12bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f93737fe640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa15390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 -- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 msgr2=0x55bf0fa12bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa12bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 -- 172.21.15.99:0/1050287276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0a8f0 msgr2=0x55bf0fa15390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa15390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 -- 172.21.15.99:0/1050287276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0f74e620 con 0x55bf0fa0b210 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f936bfff640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 0x55bf0fa12680 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f935c003fd0 tx=0x7f935c02a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89d6ba76b9ba612 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f9352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f9352ffd640 1 -- 172.21.15.99:0/1050287276 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c049070 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f9352ffd640 1 -- 172.21.15.99:0/1050287276 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f935c0c6460 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.350+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c01c440 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f9352ffd640 1 -- 172.21.15.99:0/1050287276 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f935c0c9ce0 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 10 client.16644 fetch_fsmap learned FSMap version 39 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bf0fa24090 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f9352ffd640 1 -- 172.21.15.99:0/1050287276 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c01c440 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f9352ffd640 1 -- 172.21.15.99:0/1050287276 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f935c0ce120 con 0x55bf0fa0b210 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 10 client.16644 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 20 client.16644 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa25590 0x55bf0fa27a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 4 client.16644 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0f97fb80 con 0x55bf0fa25590 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.351+0000 7f93737fe640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa25590 0x55bf0fa27a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.352+0000 7f9352ffd640 10 client.16644 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.352+0000 7f93737fe640 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa25590 0x55bf0fa27a20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e681088aa8b53db server_cookie=20356ae92526c6b0 in_seq=0 out_seq=0 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.352+0000 7f9352ffd640 1 -- 172.21.15.99:0/1050287276 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf0f97fb80 con 0x55bf0fa25590 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.352+0000 7f9352ffd640 10 client.16644 handle_command_reply: tid=0 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.352+0000 7f937c96d740 1 client.16644 shutdown 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 msgr2=0x55bf0fa12680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 0x55bf0fa12680 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f935c003fd0 tx=0x7f935c02a510 comp rx=0 tx=0).stop 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 shutdown_connections 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa25590 0x55bf0fa27a20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa12bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f9352ffd640 0 client.16644 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0b210 0x55bf0fa12680 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 --2- 172.21.15.99:0/1050287276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa15390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 >> 172.21.15.99:0/1050287276 conn(0x55bf0fa06e00 msgr2=0x55bf0fa06100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 shutdown_connections 2024-05-14T17:14:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 1 -- 172.21.15.99:0/1050287276 wait complete. 2024-05-14T17:14:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.353+0000 7f937c96d740 20 client.16644 trim_cache size 0 max 16384 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 Processor -- start 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 -- start start 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa14ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa2d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93741b3190 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93741b09d0 con 0x55bf0fa15ad0 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f937c96d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93741ae840 con 0x55bf0fa0b210 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa06880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34078/0 (socket says 172.21.15.99:34078) 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 -- 172.21.15.99:0/526657391 learned_addr learned my addr 172.21.15.99:0/526657391 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 -- 172.21.15.99:0/526657391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 msgr2=0x55bf0fa14ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa14ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 -- 172.21.15.99:0/526657391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 msgr2=0x55bf0fa2d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa2d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 -- 172.21.15.99:0/526657391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0fa280a0 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.368+0000 7f93737fe640 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa06880 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9364005fa0 tx=0x7f936400d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf55b2cfc949854a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f93697fa640 1 -- 172.21.15.99:0/526657391 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936401d070 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f93697fa640 1 -- 172.21.15.99:0/526657391 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936400f420 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f93697fa640 1 -- 172.21.15.99:0/526657391 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936400e610 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 -- 172.21.15.99:0/526657391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 msgr2=0x55bf0fa06880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa06880 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9364005fa0 tx=0x7f936400d3a0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 -- 172.21.15.99:0/526657391 shutdown_connections 2024-05-14T17:14:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa2d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa0b210 0x55bf0fa14ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 --2- 172.21.15.99:0/526657391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa06880 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 -- 172.21.15.99:0/526657391 >> 172.21.15.99:0/526657391 conn(0x55bf0fa06e00 msgr2=0x55bf0fa0cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 -- 172.21.15.99:0/526657391 shutdown_connections 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 -- 172.21.15.99:0/526657391 wait complete. 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.369+0000 7f937c96d740 1 -- start start 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 Processor -- start 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0b210 0x55bf0fa1a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa1b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93741aee90 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93741b0300 con 0x55bf0fa0b210 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93741af560 con 0x55bf0fa15ad0 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f937c96d740 10 client.0 _ll_register_callbacks cb 0x7f935c0c6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa1e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34084/0 (socket says 172.21.15.99:34084) 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 -- 172.21.15.99:0/170335526 learned_addr learned my addr 172.21.15.99:0/170335526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 -- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 msgr2=0x55bf0fa1b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa1b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 -- 172.21.15.99:0/170335526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0b210 msgr2=0x55bf0fa1a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0b210 0x55bf0fa1a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.370+0000 7f93737fe640 1 -- 172.21.15.99:0/170335526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0fa98700 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f93737fe640 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa1e110 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f936400f800 tx=0x7f936400f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa0f1ec805255f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f9352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f9352ffd640 1 -- 172.21.15.99:0/170335526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936401d050 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f9352ffd640 1 -- 172.21.15.99:0/170335526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f93640196d0 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936400f420 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f9352ffd640 1 -- 172.21.15.99:0/170335526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f93640198d0 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 10 client.16650 fetch_fsmap learned FSMap version 39 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bf0fa99910 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f9352ffd640 1 -- 172.21.15.99:0/170335526 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936400f420 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f9352ffd640 1 -- 172.21.15.99:0/170335526 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f936402a410 con 0x55bf0fa0a8f0 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 10 client.16650 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 20 client.16650 populate_metadata read hostname 'smithi099' 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa9a060 0x55bf0fa9c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 4 client.16650 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf0f979360 con 0x55bf0fa9a060 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.371+0000 7f936bfff640 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa9a060 0x55bf0fa9c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.372+0000 7f9352ffd640 10 client.16650 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.372+0000 7f936bfff640 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa9a060 0x55bf0fa9c480 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64e532a87363619 server_cookie=f6b9205f8d180951 in_seq=0 out_seq=0 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f9352ffd640 1 -- 172.21.15.99:0/170335526 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bf0f979360 con 0x55bf0fa9a060 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f9352ffd640 10 client.16650 handle_command_reply: tid=0 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 client.16650 shutdown 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 msgr2=0x55bf0fa1e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa1e110 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f936400f800 tx=0x7f936400f830 comp rx=0 tx=0).stop 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 shutdown_connections 2024-05-14T17:14:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bf0fa9a060 0x55bf0fa9c480 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf0fa15ad0 0x55bf0fa1b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f9352ffd640 0 client.16650 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf0fa0b210 0x55bf0fa1a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 --2- 172.21.15.99:0/170335526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf0fa0a8f0 0x55bf0fa1e110 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 >> 172.21.15.99:0/170335526 conn(0x55bf0fa06e00 msgr2=0x55bf0fa05870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 shutdown_connections 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 1 -- 172.21.15.99:0/170335526 wait complete. 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.432+0000 7f937c96d740 20 client.16650 trim_cache size 0 max 16384 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:14:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:14:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:14:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f93600b2a10 msgr2=0x7f93600b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f93600b2a10 0x7f93600b4ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f936c011010 tx=0x7f936c01bd40 comp rx=0 tx=0).stop 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 msgr2=0x7f9374199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f9374199710 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f935c005940 tx=0x7f935c0042f0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 shutdown_connections 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f93600b2a10 0x7f93600b4ed0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f937410d020 0x7f937419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9374104600 0x7f9374199710 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 --2- 172.21.15.99:0/1925446075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9374103ce0 0x7f93741991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 >> 172.21.15.99:0/1925446075 conn(0x7f9374078ad0 msgr2=0x7f93740fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 shutdown_connections 2024-05-14T17:14:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:14:48.433+0000 7f9379c54640 1 -- 172.21.15.99:0/1925446075 wait complete. 2024-05-14T17:14:48.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 4.20 MB/sec execute 420 sec latency 2211.267 ms 2024-05-14T17:14:49.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:48 smithi099 ceph-mon[31851]: pgmap v350: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 954 KiB/s wr, 79 op/s 2024-05-14T17:14:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:48 smithi193 ceph-mon[35962]: pgmap v350: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 954 KiB/s wr, 79 op/s 2024-05-14T17:14:49.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:48 smithi150 ceph-mon[36204]: pgmap v350: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 954 KiB/s wr, 79 op/s 2024-05-14T17:14:49.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 463595 4.19 MB/sec execute 421 sec latency 3211.357 ms 2024-05-14T17:14:50.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 465771 4.22 MB/sec execute 422 sec latency 3262.011 ms 2024-05-14T17:14:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:50 smithi099 ceph-mon[31851]: pgmap v351: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-14T17:14:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:50 smithi193 ceph-mon[35962]: pgmap v351: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-14T17:14:51.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:50 smithi150 ceph-mon[36204]: pgmap v351: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-05-14T17:14:51.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 4.22 MB/sec execute 423 sec latency 183.451 ms 2024-05-14T17:14:52.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 4.21 MB/sec execute 424 sec latency 1183.583 ms 2024-05-14T17:14:53.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:52 smithi099 ceph-mon[31851]: pgmap v352: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 839 KiB/s wr, 105 op/s 2024-05-14T17:14:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:52 smithi193 ceph-mon[35962]: pgmap v352: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 839 KiB/s wr, 105 op/s 2024-05-14T17:14:53.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:52 smithi150 ceph-mon[36204]: pgmap v352: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 839 KiB/s wr, 105 op/s 2024-05-14T17:14:53.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 4.20 MB/sec execute 425 sec latency 2183.707 ms 2024-05-14T17:14:54.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 467138 4.19 MB/sec execute 426 sec latency 3183.815 ms 2024-05-14T17:14:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:54 smithi099 ceph-mon[31851]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-14T17:14:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:54 smithi193 ceph-mon[35962]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-14T17:14:55.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:54 smithi150 ceph-mon[36204]: pgmap v353: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.4 MiB/s wr, 153 op/s 2024-05-14T17:14:55.554 INFO:tasks.workunit.client.0.smithi099.stdout: 1 469354 4.21 MB/sec execute 427 sec latency 3230.770 ms 2024-05-14T17:14:56.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 4.22 MB/sec execute 428 sec latency 246.327 ms 2024-05-14T17:14:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:56 smithi099 ceph-mon[31851]: pgmap v354: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 927 KiB/s wr, 113 op/s 2024-05-14T17:14:57.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:56 smithi193 ceph-mon[35962]: pgmap v354: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 927 KiB/s wr, 113 op/s 2024-05-14T17:14:57.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:56 smithi150 ceph-mon[36204]: pgmap v354: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 927 KiB/s wr, 113 op/s 2024-05-14T17:14:57.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 4.21 MB/sec execute 429 sec latency 1246.402 ms 2024-05-14T17:14:58.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 4.20 MB/sec execute 430 sec latency 2246.500 ms 2024-05-14T17:14:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:58 smithi099 ceph-mon[31851]: pgmap v355: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 927 KiB/s wr, 113 op/s 2024-05-14T17:14:59.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:58 smithi193 ceph-mon[35962]: pgmap v355: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 927 KiB/s wr, 113 op/s 2024-05-14T17:14:59.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:58 smithi150 ceph-mon[36204]: pgmap v355: 97 pgs: 97 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 927 KiB/s wr, 113 op/s 2024-05-14T17:14:59.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 470685 4.19 MB/sec execute 431 sec latency 3246.569 ms 2024-05-14T17:15:00.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:00.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:00.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:00.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:14:59 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:15:00.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:00.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:00.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:00.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:14:59 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:15:00.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:00.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:00.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:00.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:14:59 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:15:00.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 472612 4.21 MB/sec execute 432 sec latency 3257.590 ms 2024-05-14T17:15:01.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:00 smithi099 ceph-mon[31851]: pgmap v356: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-14T17:15:01.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:00 smithi193 ceph-mon[35962]: pgmap v356: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-14T17:15:01.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:00 smithi150 ceph-mon[36204]: pgmap v356: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 -- 172.21.15.99:0/7584334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a410f410 msgr2=0x7f15a4111800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 --2- 172.21.15.99:0/7584334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a410f410 0x7f15a4111800 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f159c009ca0 tx=0x7f159c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 -- 172.21.15.99:0/7584334 shutdown_connections 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 --2- 172.21.15.99:0/7584334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a410f410 0x7f15a4111800 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 --2- 172.21.15.99:0/7584334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a4102180 0x7f15a410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 --2- 172.21.15.99:0/7584334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4101860 0x7f15a4101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 -- 172.21.15.99:0/7584334 >> 172.21.15.99:0/7584334 conn(0x7f15a4078ad0 msgr2=0x7f15a4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 -- 172.21.15.99:0/7584334 shutdown_connections 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.526+0000 7f15aae99640 1 -- 172.21.15.99:0/7584334 wait complete. 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 Processor -- start 2024-05-14T17:15:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 -- start start 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4101860 0x7f15a419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 0x7f15a419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a410f410 0x7f15a419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4114ef0 con 0x7f15a4102180 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4114d70 con 0x7f15a4101860 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15aae99640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4115070 con 0x7f15a410f410 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 0x7f15a419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 0x7f15a419b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52488/0 (socket says 172.21.15.99:52488) 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15a3fff640 1 -- 172.21.15.99:0/3392275821 learned_addr learned my addr 172.21.15.99:0/3392275821 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15a9436640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a410f410 0x7f15a419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.527+0000 7f15a8c35640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4101860 0x7f15a419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a3fff640 1 -- 172.21.15.99:0/3392275821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a410f410 msgr2=0x7f15a419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a3fff640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a410f410 0x7f15a419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a3fff640 1 -- 172.21.15.99:0/3392275821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4101860 msgr2=0x7f15a419b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a3fff640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4101860 0x7f15a419b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a3fff640 1 -- 172.21.15.99:0/3392275821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a41a0450 con 0x7f15a4102180 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a3fff640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 0x7f15a419b9f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f159000e9f0 tx=0x7f159000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a1ffb640 1 -- 172.21.15.99:0/3392275821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159000cd80 con 0x7f15a4102180 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a41a0740 con 0x7f15a4102180 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a1ffb640 1 -- 172.21.15.99:0/3392275821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15900054d0 con 0x7f15a4102180 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15a1ffb640 1 -- 172.21.15.99:0/3392275821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590010690 con 0x7f15a4102180 2024-05-14T17:15:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.528+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41a7fd0 con 0x7f15a4102180 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.530+0000 7f15a1ffb640 1 -- 172.21.15.99:0/3392275821 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1590004e90 con 0x7f15a4102180 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 Processor -- start 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 -- start start 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2e8f0 0x55763af2ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 0x55763af39590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55763aea1c30 con 0x55763af2f210 2024-05-14T17:15:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55763ae917d0 con 0x55763af2e8f0 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15adbba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55763ae95810 con 0x55763af39ad0 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 0x55763af39590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 0x55763af39590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52504/0 (socket says 172.21.15.99:52504) 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 -- 172.21.15.99:0/1427073918 learned_addr learned my addr 172.21.15.99:0/1427073918 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a9436640 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2e8f0 0x55763af2ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a3fff640 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a1ffb640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f15940b2a10 0x7f15940b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a9436640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f15940b2a10 0x7f15940b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 -- 172.21.15.99:0/1427073918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 msgr2=0x55763af3bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 -- 172.21.15.99:0/1427073918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2e8f0 msgr2=0x55763af2ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2e8f0 0x55763af2ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a1ffb640 1 -- 172.21.15.99:0/3392275821 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f15900e2970 con 0x7f15a4102180 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a8c35640 1 -- 172.21.15.99:0/1427073918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55763aba6210 con 0x55763af2f210 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.531+0000 7f15a9436640 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2e8f0 0x55763af2ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.532+0000 7f15a8c35640 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 0x55763af39590 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1598009cf0 tx=0x7f1598011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3fb5a3225f9aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.532+0000 7f15a9436640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f15940b2a10 0x7f15940b4ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f159c02f4c0 tx=0x7f159c02a8c0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.532+0000 7f158effd640 1 -- 172.21.15.99:0/1427073918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598004230 con 0x55763af2f210 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.532+0000 7f158effd640 1 -- 172.21.15.99:0/1427073918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15980043d0 con 0x55763af2f210 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.532+0000 7f158effd640 1 -- 172.21.15.99:0/1427073918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598019880 con 0x55763af2f210 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- 172.21.15.99:0/1427073918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 msgr2=0x55763af39590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 0x55763af39590 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1598009cf0 tx=0x7f1598011810 comp rx=0 tx=0).stop 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- 172.21.15.99:0/1427073918 shutdown_connections 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2f210 0x55763af39590 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- 172.21.15.99:0/1427073918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2e8f0 0x55763af2ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- 172.21.15.99:0/1427073918 >> 172.21.15.99:0/1427073918 conn(0x55763af2ae00 msgr2=0x55763af2b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- 172.21.15.99:0/1427073918 shutdown_connections 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- 172.21.15.99:0/1427073918 wait complete. 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- start start 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 Processor -- start 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af2f210 0x55763af366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af39ad0 0x55763af36c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55763ab77790 con 0x55763af2e8f0 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55763ae4d890 con 0x55763af39ad0 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55763af3c7a0 con 0x55763af2f210 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.533+0000 7f15adbba740 10 client.0 _ll_register_callbacks cb 0x55763ae50d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52514/0 (socket says 172.21.15.99:52514) 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 -- 172.21.15.99:0/204671912 learned_addr learned my addr 172.21.15.99:0/204671912 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a3fff640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af2f210 0x55763af366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a9436640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af39ad0 0x55763af36c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 -- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af2f210 msgr2=0x55763af366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af2f210 0x55763af366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 -- 172.21.15.99:0/204671912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af39ad0 msgr2=0x55763af36c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af39ad0 0x55763af36c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 -- 172.21.15.99:0/204671912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763ac72620 con 0x55763af2e8f0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15a8c35640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af393f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f15980038d0 tx=0x7f15980045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c317dd221b11aa6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f1583fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f1583fff640 1 -- 172.21.15.99:0/204671912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598027070 con 0x55763af2e8f0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f1583fff640 1 -- 172.21.15.99:0/204671912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f1598005240 con 0x55763af2e8f0 2024-05-14T17:15:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15980043d0 con 0x55763af2e8f0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f1583fff640 1 -- 172.21.15.99:0/204671912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1598033050 con 0x55763af2e8f0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15adbba740 10 client.16662 fetch_fsmap learned FSMap version 39 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55763af48170 con 0x55763af2e8f0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.534+0000 7f1583fff640 1 -- 172.21.15.99:0/204671912 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15980043d0 con 0x55763af2e8f0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f1583fff640 1 -- 172.21.15.99:0/204671912 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f1598004b70 con 0x55763af2e8f0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15adbba740 10 client.16662 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15adbba740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15adbba740 20 client.16662 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15adbba740 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763af49580 0x55763af4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15adbba740 4 client.16662 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55763aea3b80 con 0x55763af49580 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15a3fff640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763af49580 0x55763af4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f1583fff640 10 client.16662 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.535+0000 7f15a3fff640 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763af49580 0x55763af4b970 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9096864ad275bb9 server_cookie=aaf1d411296ebc73 in_seq=0 out_seq=0 2024-05-14T17:15:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f1583fff640 1 -- 172.21.15.99:0/204671912 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55763aea3b80 con 0x55763af49580 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f1583fff640 10 client.16662 handle_command_reply: tid=0 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 client.16662 shutdown 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 msgr2=0x55763af393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af393f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f15980038d0 tx=0x7f15980045f0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 shutdown_connections 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763af49580 0x55763af4b970 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af39ad0 0x55763af36c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f1583fff640 0 client.16662 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af2f210 0x55763af366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 --2- 172.21.15.99:0/204671912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af393f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.536+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 >> 172.21.15.99:0/204671912 conn(0x55763af2ae00 msgr2=0x55763af2a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.537+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 shutdown_connections 2024-05-14T17:15:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.537+0000 7f15adbba740 1 -- 172.21.15.99:0/204671912 wait complete. 2024-05-14T17:15:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.537+0000 7f15adbba740 20 client.16662 trim_cache size 0 max 16384 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 Processor -- start 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 -- start start 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af35f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af45670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af51150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b31e0 con 0x55763af2e8f0 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b0a20 con 0x55763af2f210 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15adbba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ae890 con 0x55763af39ad0 2024-05-14T17:15:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af35f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af35f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52518/0 (socket says 172.21.15.99:52518) 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 -- 172.21.15.99:0/1025212037 learned_addr learned my addr 172.21.15.99:0/1025212037 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a9436640 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af51150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 -- 172.21.15.99:0/1025212037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 msgr2=0x55763af51150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af51150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 -- 172.21.15.99:0/1025212037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 msgr2=0x55763af45670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af45670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 -- 172.21.15.99:0/1025212037 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55763af2e0f0 con 0x55763af2e8f0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f15a8c35640 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af35f50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1598000c00 tx=0x7f1598005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac7ec310ba9c379 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f158e7fc640 1 -- 172.21.15.99:0/1025212037 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598030020 con 0x55763af2e8f0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f158e7fc640 1 -- 172.21.15.99:0/1025212037 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1598003900 con 0x55763af2e8f0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.552+0000 7f158e7fc640 1 -- 172.21.15.99:0/1025212037 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598027070 con 0x55763af2e8f0 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 -- 172.21.15.99:0/1025212037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 msgr2=0x55763af35f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af35f50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1598000c00 tx=0x7f1598005970 comp rx=0 tx=0).stop 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 -- 172.21.15.99:0/1025212037 shutdown_connections 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af51150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af45670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 --2- 172.21.15.99:0/1025212037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af35f50 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 -- 172.21.15.99:0/1025212037 >> 172.21.15.99:0/1025212037 conn(0x55763af2ae00 msgr2=0x55763af35840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 -- 172.21.15.99:0/1025212037 shutdown_connections 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 -- 172.21.15.99:0/1025212037 wait complete. 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 -- start start 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.553+0000 7f15adbba740 1 Processor -- start 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af41930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af41e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41aeee0 con 0x55763af2e8f0 2024-05-14T17:15:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b0350 con 0x55763af2f210 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41af5b0 con 0x55763af39ad0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 10 client.0 _ll_register_callbacks cb 0x7f1598009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af41930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af41930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52526/0 (socket says 172.21.15.99:52526) 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 -- 172.21.15.99:0/2151734376 learned_addr learned my addr 172.21.15.99:0/2151734376 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a3fff640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af41e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 -- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 msgr2=0x55763af3ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a9436640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 -- 172.21.15.99:0/2151734376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 msgr2=0x55763af41e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af41e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 -- 172.21.15.99:0/2151734376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763af3f550 con 0x55763af2e8f0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a3fff640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af41e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15a8c35640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af41930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1598011f50 tx=0x7f1598019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b3fd07ae84bd1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f1583fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f1583fff640 1 -- 172.21.15.99:0/2151734376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598030030 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f1583fff640 1 -- 172.21.15.99:0/2151734376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f1598019c20 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.554+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1598003900 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f1583fff640 1 -- 172.21.15.99:0/2151734376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1598027040 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 10 client.16668 fetch_fsmap learned FSMap version 39 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55763afb3820 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f1583fff640 1 -- 172.21.15.99:0/2151734376 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1598003900 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f1583fff640 1 -- 172.21.15.99:0/2151734376 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f1598004740 con 0x55763af2e8f0 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 10 client.16668 fetch_fsmap finished waiting for FSMap version 39 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 20 client.16668 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763afb3e50 0x55763afb6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 4 client.16668 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55763ae9d360 con 0x55763afb3e50 2024-05-14T17:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.555+0000 7f15a3fff640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763afb3e50 0x55763afb6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:01.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 4.21 MB/sec execute 433 sec latency 37.331 ms 2024-05-14T17:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.556+0000 7f15a3fff640 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763afb3e50 0x55763afb6270 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efc2e4ca0973b1b server_cookie=dda88d6ad31a15c9 in_seq=0 out_seq=0 2024-05-14T17:15:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.556+0000 7f1583fff640 10 client.16668 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f1583fff640 1 -- 172.21.15.99:0/2151734376 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55763ae9d360 con 0x55763afb3e50 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f1583fff640 10 client.16668 handle_command_reply: tid=0 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 client.16668 shutdown 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 msgr2=0x55763af41930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af41930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1598011f50 tx=0x7f1598019420 comp rx=0 tx=0).stop 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 shutdown_connections 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55763afb3e50 0x55763afb6270 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55763af39ad0 0x55763af3ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55763af2f210 0x55763af41e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 --2- 172.21.15.99:0/2151734376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55763af2e8f0 0x55763af41930 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f1583fff640 0 client.16668 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 >> 172.21.15.99:0/2151734376 conn(0x55763af2ae00 msgr2=0x55763af35f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 shutdown_connections 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 1 -- 172.21.15.99:0/2151734376 wait complete. 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.607+0000 7f15adbba740 20 client.16668 trim_cache size 0 max 16384 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:15:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:15:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f15940b2a10 msgr2=0x7f15940b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f15940b2a10 0x7f15940b4ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f159c02f4c0 tx=0x7f159c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 msgr2=0x7f15a419b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 0x7f15a419b9f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f159000e9f0 tx=0x7f159000edf0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 shutdown_connections 2024-05-14T17:15:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f15940b2a10 0x7f15940b4ed0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a410f410 0x7f15a419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15a4102180 0x7f15a419b9f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 --2- 172.21.15.99:0/3392275821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4101860 0x7f15a419b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 >> 172.21.15.99:0/3392275821 conn(0x7f15a4078ad0 msgr2=0x7f15a4110900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 shutdown_connections 2024-05-14T17:15:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:01.608+0000 7f15aae99640 1 -- 172.21.15.99:0/3392275821 wait complete. 2024-05-14T17:15:02.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 4.20 MB/sec execute 434 sec latency 1037.388 ms 2024-05-14T17:15:03.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:02 smithi099 ceph-mon[31851]: pgmap v357: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 852 KiB/s wr, 82 op/s 2024-05-14T17:15:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:02 smithi193 ceph-mon[35962]: pgmap v357: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 852 KiB/s wr, 82 op/s 2024-05-14T17:15:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:02 smithi150 ceph-mon[36204]: pgmap v357: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 852 KiB/s wr, 82 op/s 2024-05-14T17:15:03.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 4.19 MB/sec execute 435 sec latency 2037.458 ms 2024-05-14T17:15:04.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 474231 4.18 MB/sec execute 436 sec latency 3037.538 ms 2024-05-14T17:15:05.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:05 smithi099 ceph-mon[31851]: pgmap v358: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-05-14T17:15:05.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:05 smithi193 ceph-mon[35962]: pgmap v358: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-05-14T17:15:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:05 smithi150 ceph-mon[36204]: pgmap v358: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-05-14T17:15:05.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 476485 4.20 MB/sec execute 437 sec latency 3083.379 ms 2024-05-14T17:15:06.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 4.21 MB/sec execute 438 sec latency 196.542 ms 2024-05-14T17:15:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:07 smithi099 ceph-mon[31851]: pgmap v359: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 769 KiB/s wr, 146 op/s 2024-05-14T17:15:07.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:07 smithi099 ceph-mon[31851]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:15:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:07 smithi193 ceph-mon[35962]: pgmap v359: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 769 KiB/s wr, 146 op/s 2024-05-14T17:15:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:07 smithi193 ceph-mon[35962]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:15:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:07 smithi150 ceph-mon[36204]: pgmap v359: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 769 KiB/s wr, 146 op/s 2024-05-14T17:15:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:07 smithi150 ceph-mon[36204]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:15:07.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 4.20 MB/sec execute 439 sec latency 1196.646 ms 2024-05-14T17:15:08.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 4.19 MB/sec execute 440 sec latency 2196.747 ms 2024-05-14T17:15:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:09 smithi099 ceph-mon[31851]: pgmap v361: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 922 KiB/s wr, 175 op/s 2024-05-14T17:15:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:09 smithi193 ceph-mon[35962]: pgmap v361: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 922 KiB/s wr, 175 op/s 2024-05-14T17:15:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:09 smithi150 ceph-mon[36204]: pgmap v361: 97 pgs: 97 active+clean; 121 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 922 KiB/s wr, 175 op/s 2024-05-14T17:15:09.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 477780 4.18 MB/sec execute 441 sec latency 3196.845 ms 2024-05-14T17:15:10.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 480044 4.20 MB/sec execute 442 sec latency 3244.519 ms 2024-05-14T17:15:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:11 smithi193 ceph-mon[35962]: pgmap v362: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 932 KiB/s wr, 175 op/s 2024-05-14T17:15:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:11 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:11 smithi150 ceph-mon[36204]: pgmap v362: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 932 KiB/s wr, 175 op/s 2024-05-14T17:15:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:11 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:11.475 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T17:15:11.475 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.65 2024-05-14T17:15:11.475 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:15:11.498 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:11 smithi099 ceph-mon[31851]: pgmap v362: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 932 KiB/s wr, 175 op/s 2024-05-14T17:15:11.498 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:11 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:11.555 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 4.20 MB/sec execute 443 sec latency 228.488 ms 2024-05-14T17:15:11.713 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:12.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 4.19 MB/sec execute 444 sec latency 1228.592 ms 2024-05-14T17:15:13.153 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:13 smithi099 ceph-mon[31851]: pgmap v363: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 932 KiB/s wr, 175 op/s 2024-05-14T17:15:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:13 smithi193 ceph-mon[35962]: pgmap v363: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 932 KiB/s wr, 175 op/s 2024-05-14T17:15:13.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 -- 172.21.15.99:0/1247265475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 msgr2=0x7f3960103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1247265475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f3960103bd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f39500099c0 tx=0x7f3950009cd0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 -- 172.21.15.99:0/1247265475 shutdown_connections 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1247265475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39601088f0 0x7f3960110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1247265475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f3960103bd0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1247265475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 0x7f39601081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 -- 172.21.15.99:0/1247265475 >> 172.21.15.99:0/1247265475 conn(0x7f396007a6a0 msgr2=0x7f396007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 -- 172.21.15.99:0/1247265475 shutdown_connections 2024-05-14T17:15:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.383+0000 7f3967e0e700 1 -- 172.21.15.99:0/1247265475 wait complete. 2024-05-14T17:15:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 Processor -- start 2024-05-14T17:15:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 -- start start 2024-05-14T17:15:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f396019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 0x7f396019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:13.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39601088f0 0x7f3960199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39601136e0 con 0x7f39601037f0 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3960113560 con 0x7f39601088f0 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3967e0e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3960113860 con 0x7f39601041a0 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f396019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f396019f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40844/0 (socket says 172.21.15.99:40844) 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 -- 172.21.15.99:0/1120216914 learned_addr learned my addr 172.21.15.99:0/1120216914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f39653a9700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 0x7f396019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 -- 172.21.15.99:0/1120216914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 msgr2=0x7f396019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 0x7f396019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f39663ab700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39601088f0 0x7f3960199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 -- 172.21.15.99:0/1120216914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39601088f0 msgr2=0x7f3960199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:13.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39601088f0 0x7f3960199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 -- 172.21.15.99:0/1120216914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396019a180 con 0x7f39601037f0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f39653a9700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 0x7f396019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.384+0000 7f3965baa700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f396019f860 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3950006010 tx=0x7f3950004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.385+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950026070 con 0x7f39601037f0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.385+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f395000f460 con 0x7f39601037f0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.385+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396019a410 con 0x7f39601037f0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.385+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950009e60 con 0x7f39601037f0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.385+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601a6730 con 0x7f39601037f0 2024-05-14T17:15:13.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.386+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396006a230 con 0x7f39601037f0 2024-05-14T17:15:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.387+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3950029070 con 0x7f39601037f0 2024-05-14T17:15:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.387+0000 7f3956ffd700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f394c0b2a70 0x7f394c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.388+0000 7f39653a9700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f394c0b2a70 0x7f394c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.388+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7f39500e30b0 con 0x7f39601037f0 2024-05-14T17:15:13.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.388+0000 7f39653a9700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f394c0b2a70 0x7f394c0b4f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3960103180 tx=0x7f395c009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:13.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.391+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3950030080 con 0x7f39601037f0 2024-05-14T17:15:13.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:13 smithi150 ceph-mon[36204]: pgmap v363: 97 pgs: 97 active+clean; 122 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 932 KiB/s wr, 175 op/s 2024-05-14T17:15:13.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 4.18 MB/sec execute 445 sec latency 2228.658 ms 2024-05-14T17:15:13.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.650+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f396019af50 con 0x7f39601037f0 2024-05-14T17:15:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.651+0000 7f3956ffd700 1 -- 172.21.15.99:0/1120216914 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11875 (secure 0 0 0) 0x7f3950013070 con 0x7f39601037f0 2024-05-14T17:15:13.650 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:15:13.651 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:15:10.007230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24448,24463,24443]},"id":1}]} 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f394c0b2a70 msgr2=0x7f394c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f394c0b2a70 0x7f394c0b4f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3960103180 tx=0x7f395c009260 comp rx=0 tx=0).stop 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 msgr2=0x7f396019f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f396019f860 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3950006010 tx=0x7f3950004690 comp rx=0 tx=0).stop 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 shutdown_connections 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f394c0b2a70 0x7f394c0b4f30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39601088f0 0x7f3960199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39601037f0 0x7f396019f860 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 --2- 172.21.15.99:0/1120216914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39601041a0 0x7f396019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:13.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 >> 172.21.15.99:0/1120216914 conn(0x7f396007a6a0 msgr2=0x7f3960109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:13.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 shutdown_connections 2024-05-14T17:15:13.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:13.653+0000 7f3967e0e700 1 -- 172.21.15.99:0/1120216914 wait complete. 2024-05-14T17:15:13.654 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 40 2024-05-14T17:15:14.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:14 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1120216914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:14.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:14 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1120216914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:14.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:14 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1120216914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:14.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 481360 4.17 MB/sec execute 446 sec latency 3228.690 ms 2024-05-14T17:15:14.584 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0' 2024-05-14T17:15:14.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 -- 172.21.15.99:0/3832643493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08101980 msgr2=0x7fbe08101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/3832643493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08101980 0x7fbe08101e00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00fb60 tx=0x7fbdfc02e4b0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 -- 172.21.15.99:0/3832643493 shutdown_connections 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/3832643493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe0810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/3832643493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08101980 0x7fbe08101e00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/3832643493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08107940 0x7fbe08107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 -- 172.21.15.99:0/3832643493 >> 172.21.15.99:0/3832643493 conn(0x7fbe08078ad0 msgr2=0x7fbe08078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 -- 172.21.15.99:0/3832643493 shutdown_connections 2024-05-14T17:15:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 -- 172.21.15.99:0/3832643493 wait complete. 2024-05-14T17:15:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.708+0000 7fbe0dad1640 1 Processor -- start 2024-05-14T17:15:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 -- start start 2024-05-14T17:15:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08101980 0x7fbe08196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe08197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08107940 0x7fbe0819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08110af0 con 0x7fbe08101980 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08110970 con 0x7fbe08107940 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe0dad1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08110c70 con 0x7fbe08102340 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe08197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08101980 0x7fbe08196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe08197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.99:55080/0 (socket says 172.21.15.99:55080) 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe06ffd640 1 -- 172.21.15.99:0/2785882059 learned_addr learned my addr 172.21.15.99:0/2785882059 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.709+0000 7fbe07fff640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08107940 0x7fbe0819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe06ffd640 1 -- 172.21.15.99:0/2785882059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08107940 msgr2=0x7fbe0819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe06ffd640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08107940 0x7fbe0819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe06ffd640 1 -- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08101980 msgr2=0x7fbe08196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe06ffd640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08101980 0x7fbe08196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe06ffd640 1 -- 172.21.15.99:0/2785882059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0819bfc0 con 0x7fbe08102340 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe07fff640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08107940 0x7fbe0819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe077fe640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08101980 0x7fbe08196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe06ffd640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe08197510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00fc80 tx=0x7fbdfc034900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe04ff9640 1 -- 172.21.15.99:0/2785882059 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc044070 con 0x7fbe08102340 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe04ff9640 1 -- 172.21.15.99:0/2785882059 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdfc0358b0 con 0x7fbe08102340 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0819c250 con 0x7fbe08102340 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe04ff9640 1 -- 172.21.15.99:0/2785882059 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc004ad0 con 0x7fbe08102340 2024-05-14T17:15:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.710+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081a3b40 con 0x7fbe08102340 2024-05-14T17:15:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.713+0000 7fbe04ff9640 1 -- 172.21.15.99:0/2785882059 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbdfc03d040 con 0x7fbe08102340 2024-05-14T17:15:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.713+0000 7fbe04ff9640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbdf00b2a90 0x7fbdf00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.713+0000 7fbe077fe640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbdf00b2a90 0x7fbdf00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.713+0000 7fbe04ff9640 1 -- 172.21.15.99:0/2785882059 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fbdfc1071f0 con 0x7fbe08102340 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 Processor -- start 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe077fe640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbdf00b2a90 0x7fbdf00b4f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005fd0 tx=0x7fbdf8009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 -- start start 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc2b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 0x564bbc2c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bbc228c30 con 0x564bbc2b6210 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564bbc2187d0 con 0x564bbc2b58f0 2024-05-14T17:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe107eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564bbc21c810 con 0x564bbc2c0ad0 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 0x564bbc2c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 0x564bbc2c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50056/0 (socket says 172.21.15.99:50056) 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc2b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe06ffd640 1 -- 172.21.15.99:0/2705884019 learned_addr learned my addr 172.21.15.99:0/2705884019 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.714+0000 7fbe07fff640 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe077fe640 1 -- 172.21.15.99:0/2705884019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 msgr2=0x564bbc2c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe077fe640 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe077fe640 1 -- 172.21.15.99:0/2705884019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 msgr2=0x564bbc2c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe077fe640 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 0x564bbc2c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe077fe640 1 -- 172.21.15.99:0/2705884019 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbbf2d210 con 0x564bbc2b58f0 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe06ffd640 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 0x564bbc2c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbe077fe640 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc2b5cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e670 tx=0x7fbdf8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b175ba0c0a3eaeb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbdedffb640 1 -- 172.21.15.99:0/2705884019 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8005220 con 0x564bbc2b58f0 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbdedffb640 1 -- 172.21.15.99:0/2705884019 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf8004c50 con 0x564bbc2b58f0 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.715+0000 7fbdedffb640 1 -- 172.21.15.99:0/2705884019 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf801d9f0 con 0x564bbc2b58f0 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 -- 172.21.15.99:0/2705884019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 msgr2=0x564bbc2b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc2b5cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e670 tx=0x7fbdf8014000 comp rx=0 tx=0).stop 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 -- 172.21.15.99:0/2705884019 shutdown_connections 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b6210 0x564bbc2c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 --2- 172.21.15.99:0/2705884019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc2b5cd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 -- 172.21.15.99:0/2705884019 >> 172.21.15.99:0/2705884019 conn(0x564bbc2b1e00 msgr2=0x564bbc2b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 -- 172.21.15.99:0/2705884019 shutdown_connections 2024-05-14T17:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 -- 172.21.15.99:0/2705884019 wait complete. 2024-05-14T17:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.716+0000 7fbe107eb740 1 -- start start 2024-05-14T17:15:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 Processor -- start 2024-05-14T17:15:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b58f0 0x564bbc2c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2bdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe07fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2bdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b58f0 0x564bbc2c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:55354/0 (socket says 172.21.15.99:55354) 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe06ffd640 1 -- 172.21.15.99:0/4148127834 learned_addr learned my addr 172.21.15.99:0/4148127834 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bbbefe790 con 0x564bbc2c0ad0 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564bbc1d4890 con 0x564bbc2b6210 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564bbc2c37a0 con 0x564bbc2b58f0 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.717+0000 7fbe107eb740 10 client.0 _ll_register_callbacks cb 0x564bbc1d7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe07fff640 1 -- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b58f0 msgr2=0x564bbc2c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe07fff640 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b58f0 0x564bbc2c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe07fff640 1 -- 172.21.15.99:0/4148127834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 msgr2=0x564bbc2bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe07fff640 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe07fff640 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbbff9620 con 0x564bbc2c0ad0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe06ffd640 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe07fff640 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2bdc40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x564bbc2b2f90 tx=0x7fbe00013600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa44716d5dd7a500 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbde6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbde6ffd640 1 -- 172.21.15.99:0/4148127834 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00017630 con 0x564bbc2c0ad0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbde6ffd640 1 -- 172.21.15.99:0/4148127834 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fbe000177d0 con 0x564bbc2c0ad0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf8004c50 con 0x564bbc2c0ad0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbde6ffd640 1 -- 172.21.15.99:0/4148127834 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbe00017a10 con 0x564bbc2c0ad0 2024-05-14T17:15:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe107eb740 10 client.16686 fetch_fsmap learned FSMap version 40 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564bbc2cf1c0 con 0x564bbc2c0ad0 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.718+0000 7fbde6ffd640 1 -- 172.21.15.99:0/4148127834 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf8004c50 con 0x564bbc2c0ad0 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbde6ffd640 1 -- 172.21.15.99:0/4148127834 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fbe00026070 con 0x564bbc2c0ad0 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe107eb740 10 client.16686 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe107eb740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe107eb740 20 client.16686 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe107eb740 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc2d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe107eb740 4 client.16686 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bbc22ab80 con 0x564bbc2d0a20 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe077fe640 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc2d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbde6ffd640 10 client.16686 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.719+0000 7fbe077fe640 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc2d2e10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a63e0907cd3592 server_cookie=153a865215a72121 in_seq=0 out_seq=0 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbde6ffd640 1 -- 172.21.15.99:0/4148127834 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564bbc22ab80 con 0x564bbc2d0a20 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbde6ffd640 10 client.16686 handle_command_reply: tid=0 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 client.16686 shutdown 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 msgr2=0x564bbc2bdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2bdc40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x564bbc2b2f90 tx=0x7fbe00013600 comp rx=0 tx=0).stop 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 shutdown_connections 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc2d2e10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2bdc40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbde6ffd640 0 client.16686 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 --2- 172.21.15.99:0/4148127834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b58f0 0x564bbc2c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.720+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 >> 172.21.15.99:0/4148127834 conn(0x564bbc2b1e00 msgr2=0x564bbc2b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.721+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 shutdown_connections 2024-05-14T17:15:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.721+0000 7fbe107eb740 1 -- 172.21.15.99:0/4148127834 wait complete. 2024-05-14T17:15:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.721+0000 7fbe107eb740 20 client.16686 trim_cache size 0 max 16384 2024-05-14T17:15:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.752+0000 7fbe107eb740 1 Processor -- start 2024-05-14T17:15:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.752+0000 7fbe107eb740 1 -- start start 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.753+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b58f0 0x564bbc2cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.753+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.753+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.753+0000 7fbe107eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081aed70 con 0x564bbc2b58f0 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.753+0000 7fbe107eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081ac5e0 con 0x564bbc2b6210 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.753+0000 7fbe107eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081aa450 con 0x564bbc2c0ad0 2024-05-14T17:15:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b58f0 0x564bbc2cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b58f0 0x564bbc2cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50070/0 (socket says 172.21.15.99:50070) 2024-05-14T17:15:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe077fe640 1 -- 172.21.15.99:0/1011277385 learned_addr learned my addr 172.21.15.99:0/1011277385 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe06ffd640 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 -- 172.21.15.99:0/1011277385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 msgr2=0x564bbc2d83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2d83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 -- 172.21.15.99:0/1011277385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b58f0 msgr2=0x564bbc2cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b58f0 0x564bbc2cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 -- 172.21.15.99:0/1011277385 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbc2bc6b0 con 0x564bbc2c0ad0 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbe07fff640 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2d88f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe0001fc70 tx=0x7fbe00013b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=192efa225196e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbded7fa640 1 -- 172.21.15.99:0/1011277385 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00035070 con 0x564bbc2c0ad0 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbded7fa640 1 -- 172.21.15.99:0/1011277385 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0000b840 con 0x564bbc2c0ad0 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.754+0000 7fbded7fa640 1 -- 172.21.15.99:0/1011277385 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0002c040 con 0x564bbc2c0ad0 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 -- 172.21.15.99:0/1011277385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 msgr2=0x564bbc2d88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2d88f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe0001fc70 tx=0x7fbe00013b00 comp rx=0 tx=0).stop 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 -- 172.21.15.99:0/1011277385 shutdown_connections 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2c0ad0 0x564bbc2d88f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b6210 0x564bbc2d83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 --2- 172.21.15.99:0/1011277385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2b58f0 0x564bbc2cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 -- 172.21.15.99:0/1011277385 >> 172.21.15.99:0/1011277385 conn(0x564bbc2b1e00 msgr2=0x564bbc2b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 -- 172.21.15.99:0/1011277385 shutdown_connections 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.755+0000 7fbe107eb740 1 -- 172.21.15.99:0/1011277385 wait complete. 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 -- start start 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 Processor -- start 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc3332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b6210 0x564bbc333820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081aaaa0 con 0x564bbc2c0ad0 2024-05-14T17:15:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081abf10 con 0x564bbc2b58f0 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081ab170 con 0x564bbc2b6210 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe107eb740 10 client.0 _ll_register_callbacks cb 0x7fbe0000a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe07fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe06ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b6210 0x564bbc333820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc3332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe07fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50076/0 (socket says 172.21.15.99:50076) 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe07fff640 1 -- 172.21.15.99:0/3362680959 learned_addr learned my addr 172.21.15.99:0/3362680959 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.756+0000 7fbe07fff640 1 -- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b6210 msgr2=0x564bbc333820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe07fff640 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b6210 0x564bbc333820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe07fff640 1 -- 172.21.15.99:0/3362680959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 msgr2=0x564bbc3332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe07fff640 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc3332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe07fff640 1 -- 172.21.15.99:0/3362680959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbc2c6530 con 0x564bbc2c0ad0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe07fff640 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c5e40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbe00000f80 tx=0x7fbe00013fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea483b92af77836 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbde6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbde6ffd640 1 -- 172.21.15.99:0/3362680959 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0002c070 con 0x564bbc2c0ad0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0000b840 con 0x564bbc2c0ad0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbde6ffd640 1 -- 172.21.15.99:0/3362680959 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8565+0+0 (secure 0 0 0) 0x7fbe0001f400 con 0x564bbc2c0ad0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbde6ffd640 1 -- 172.21.15.99:0/3362680959 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbe00035040 con 0x564bbc2c0ad0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe107eb740 10 client.16692 fetch_fsmap learned FSMap version 40 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564bbc33aa90 con 0x564bbc2c0ad0 2024-05-14T17:15:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbde6ffd640 1 -- 172.21.15.99:0/3362680959 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0000b840 con 0x564bbc2c0ad0 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbde6ffd640 1 -- 172.21.15.99:0/3362680959 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7fbe00026040 con 0x564bbc2c0ad0 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.757+0000 7fbe107eb740 10 client.16692 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe107eb740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe107eb740 20 client.16692 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe107eb740 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc33d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe107eb740 4 client.16692 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bbc224360 con 0x564bbc2d0a20 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe077fe640 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc33d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbe077fe640 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc33d110 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddb215ce36bf4d0 server_cookie=d5df24fc8271525b in_seq=0 out_seq=0 2024-05-14T17:15:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.758+0000 7fbde6ffd640 10 client.16692 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbde6ffd640 1 -- 172.21.15.99:0/3362680959 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564bbc224360 con 0x564bbc2d0a20 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbde6ffd640 10 client.16692 handle_command_reply: tid=0 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 client.16692 shutdown 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 msgr2=0x564bbc2c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c5e40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbe00000f80 tx=0x7fbe00013fb0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 shutdown_connections 2024-05-14T17:15:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564bbc2d0a20 0x564bbc33d110 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bbc2c0ad0 0x564bbc2c5e40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564bbc2b6210 0x564bbc333820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 --2- 172.21.15.99:0/3362680959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bbc2b58f0 0x564bbc3332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbde6ffd640 0 client.16692 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.791+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 >> 172.21.15.99:0/3362680959 conn(0x564bbc2b1e00 msgr2=0x564bbc2cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 shutdown_connections 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe107eb740 1 -- 172.21.15.99:0/3362680959 wait complete. 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe107eb740 20 client.16692 trim_cache size 0 max 16384 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:15:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:15:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:15:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbdf00b2a90 msgr2=0x7fbdf00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbdf00b2a90 0x7fbdf00b4f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005fd0 tx=0x7fbdf8009260 comp rx=0 tx=0).stop 2024-05-14T17:15:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 msgr2=0x7fbe08197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.792+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe08197510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00fc80 tx=0x7fbdfc034900 comp rx=0 tx=0).stop 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 shutdown_connections 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbdf00b2a90 0x7fbdf00b4f50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe08107940 0x7fbe0819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe08102340 0x7fbe08197510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 --2- 172.21.15.99:0/2785882059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe08101980 0x7fbe08196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 >> 172.21.15.99:0/2785882059 conn(0x7fbe08078ad0 msgr2=0x7fbe08104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 shutdown_connections 2024-05-14T17:15:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:14.793+0000 7fbe0dad1640 1 -- 172.21.15.99:0/2785882059 wait complete. 2024-05-14T17:15:14.927 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:15.210 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:15 smithi099 ceph-mon[31851]: pgmap v364: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-05-14T17:15:15.210 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:15 smithi099 ceph-mon[31851]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:15:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:15 smithi193 ceph-mon[35962]: pgmap v364: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-05-14T17:15:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:15 smithi193 ceph-mon[35962]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:15:15.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:15 smithi150 ceph-mon[36204]: pgmap v364: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-05-14T17:15:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:15 smithi150 ceph-mon[36204]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:15:15.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 483586 4.20 MB/sec execute 447 sec latency 3253.694 ms 2024-05-14T17:15:16.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484911 4.20 MB/sec execute 448 sec latency 277.980 ms 2024-05-14T17:15:16.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 -- 172.21.15.99:0/1312980179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80103850 msgr2=0x7ffb80103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 --2- 172.21.15.99:0/1312980179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80103850 0x7ffb80103c30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffb700099c0 tx=0x7ffb70009cd0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 -- 172.21.15.99:0/1312980179 shutdown_connections 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 --2- 172.21.15.99:0/1312980179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb80108920 0x7ffb801104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 --2- 172.21.15.99:0/1312980179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80103850 0x7ffb80103c30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 --2- 172.21.15.99:0/1312980179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80104200 0x7ffb801081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 -- 172.21.15.99:0/1312980179 >> 172.21.15.99:0/1312980179 conn(0x7ffb8007a620 msgr2=0x7ffb8007aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 -- 172.21.15.99:0/1312980179 shutdown_connections 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.736+0000 7ffb86374700 1 -- 172.21.15.99:0/1312980179 wait complete. 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 Processor -- start 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 -- start start 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80103850 0x7ffb80075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 0x7ffb80075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb80108920 0x7ffb800762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80113750 con 0x7ffb80104200 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801135d0 con 0x7ffb80103850 2024-05-14T17:15:16.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb86374700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801138d0 con 0x7ffb80108920 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 0x7ffb80075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 0x7ffb80075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50096/0 (socket says 172.21.15.99:50096) 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb77fff700 1 -- 172.21.15.99:0/4127831734 learned_addr learned my addr 172.21.15.99:0/4127831734 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80103850 0x7ffb80075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.737+0000 7ffb84911700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb80108920 0x7ffb800762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb77fff700 1 -- 172.21.15.99:0/4127831734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb80108920 msgr2=0x7ffb800762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb77fff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb80108920 0x7ffb800762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb77fff700 1 -- 172.21.15.99:0/4127831734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80103850 msgr2=0x7ffb80075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb77fff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80103850 0x7ffb80075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb77fff700 1 -- 172.21.15.99:0/4127831734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801a8080 con 0x7ffb80104200 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80103850 0x7ffb80075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb77fff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 0x7ffb80075d80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800d950 tx=0x7ffb6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb7dffb700 1 -- 172.21.15.99:0/4127831734 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68016070 con 0x7ffb80104200 2024-05-14T17:15:16.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb7dffb700 1 -- 172.21.15.99:0/4127831734 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6800fd40 con 0x7ffb80104200 2024-05-14T17:15:16.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb801a8370 con 0x7ffb80104200 2024-05-14T17:15:16.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb7dffb700 1 -- 172.21.15.99:0/4127831734 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6800e3b0 con 0x7ffb80104200 2024-05-14T17:15:16.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.738+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb801a8900 con 0x7ffb80104200 2024-05-14T17:15:16.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.740+0000 7ffb7dffb700 1 -- 172.21.15.99:0/4127831734 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffb68019070 con 0x7ffb80104200 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.741+0000 7ffb7dffb700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ffb6c0b2b20 0x7ffb6c0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.741+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ffb6c0b2b20 0x7ffb6c0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.741+0000 7ffb7dffb700 1 -- 172.21.15.99:0/4127831734 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7ffb680e1ab0 con 0x7ffb80104200 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 Processor -- start 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 -- start start 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ffb6c0b2b20 0x7ffb6c0b4fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b580 tx=0x7ffb700058e0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f8e40 0x5648029f9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 0x5648029f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564802951e30 con 0x5648029f9760 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648029419d0 con 0x5648029f8e40 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564802945a10 con 0x5648029f3b70 2024-05-14T17:15:16.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 0x5648029f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 0x5648029f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50106/0 (socket says 172.21.15.99:50106) 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 -- 172.21.15.99:0/3691093107 learned_addr learned my addr 172.21.15.99:0/3691093107 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb84911700 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f8e40 0x5648029f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 -- 172.21.15.99:0/3691093107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 msgr2=0x5648029f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 -- 172.21.15.99:0/3691093107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f8e40 msgr2=0x5648029f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f8e40 0x5648029f9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 -- 172.21.15.99:0/3691093107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564802891a70 con 0x5648029f9760 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.742+0000 7ffb77fff700 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 0x5648029f3550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800e010 tx=0x7ffb68010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6f641dc27220c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb767fc700 1 -- 172.21.15.99:0/3691093107 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68020040 con 0x5648029f9760 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb767fc700 1 -- 172.21.15.99:0/3691093107 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb680b1bf0 con 0x5648029f9760 2024-05-14T17:15:16.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb767fc700 1 -- 172.21.15.99:0/3691093107 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680a9250 con 0x5648029f9760 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/3691093107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 msgr2=0x5648029f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 0x5648029f3550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800e010 tx=0x7ffb68010000 comp rx=0 tx=0).stop 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/3691093107 shutdown_connections 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f8e40 0x5648029f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/3691093107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f9760 0x5648029f3550 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/3691093107 >> 172.21.15.99:0/3691093107 conn(0x56480291e3f0 msgr2=0x5648029edd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/3691093107 shutdown_connections 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.743+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/3691093107 wait complete. 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 -- start start 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 Processor -- start 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029fc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x5648029fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564802848140 con 0x5648029f8e40 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5648028e3960 con 0x5648029f9760 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564802a00140 con 0x5648029f3b70 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb90c0ab80 10 client.0 _ll_register_callbacks cb 0x5648028d8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029fc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50114/0 (socket says 172.21.15.99:50114) 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb77fff700 1 -- 172.21.15.99:0/2313621585 learned_addr learned my addr 172.21.15.99:0/2313621585 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb84911700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x5648029fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.744+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb77fff700 1 -- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 msgr2=0x5648029ff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029ff4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb77fff700 1 -- 172.21.15.99:0/2313621585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 msgr2=0x5648029fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x5648029fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb77fff700 1 -- 172.21.15.99:0/2313621585 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648028e0830 con 0x5648029f8e40 2024-05-14T17:15:16.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029fc870 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffb680a8cd0 tx=0x7ffb680a52b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9f670437ce3589 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2313621585 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680a5380 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2313621585 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7ffb680106a0 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb680b1bf0 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2313621585 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffb680a8ff0 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb90c0ab80 10 client.16704 fetch_fsmap learned FSMap version 40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564802a0a8c0 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2313621585 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb680b1bf0 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.745+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2313621585 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7ffb680a8080 con 0x5648029f8e40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb90c0ab80 10 client.16704 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb90c0ab80 10 client.16704 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb90c0ab80 20 client.16704 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a0cd90 0x564802a0f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb90c0ab80 4 client.16704 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:16.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648029bf500 con 0x564802a0cd90 2024-05-14T17:15:16.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a0cd90 0x564802a0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb5f7fe700 10 client.16704 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:16.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.746+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a0cd90 0x564802a0f180 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7635b3e1a6c8758 server_cookie=1cb021dc6c3dfe51 in_seq=0 out_seq=0 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2313621585 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648029bf500 con 0x564802a0cd90 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb5f7fe700 10 client.16704 handle_command_reply: tid=0 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 client.16704 shutdown 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 msgr2=0x5648029fc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029fc870 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffb680a8cd0 tx=0x7ffb680a52b0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 shutdown_connections 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x5648029fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f3b70 0x5648029ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a0cd90 0x564802a0f180 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2313621585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029fc870 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.747+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 >> 172.21.15.99:0/2313621585 conn(0x56480291e3f0 msgr2=0x5648029e8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:16.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.748+0000 7ffb5f7fe700 0 client.16704 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:16.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.748+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 shutdown_connections 2024-05-14T17:15:16.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.748+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2313621585 wait complete. 2024-05-14T17:15:16.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.748+0000 7ffb90c0ab80 20 client.16704 trim_cache size 0 max 16384 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.767+0000 7ffb90c0ab80 1 Processor -- start 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.767+0000 7ffb90c0ab80 1 -- start start 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f3b70 0x5648029f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f9760 0x5648029fc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801b2a20 con 0x5648029f8e40 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801b0250 con 0x5648029f3b70 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801ae0c0 con 0x5648029f9760 2024-05-14T17:15:16.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50130/0 (socket says 172.21.15.99:50130) 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 -- 172.21.15.99:0/2718941274 learned_addr learned my addr 172.21.15.99:0/2718941274 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 -- 172.21.15.99:0/2718941274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f9760 msgr2=0x5648029fc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb84911700 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f9760 0x5648029fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f9760 0x5648029fc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 -- 172.21.15.99:0/2718941274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f3b70 msgr2=0x5648029f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f3b70 0x5648029f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f3b70 0x5648029f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 -- 172.21.15.99:0/2718941274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648029f9470 con 0x5648029f8e40 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029e8980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffb680a8d20 tx=0x7ffb6800fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad49fcc4f3be9ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb75ffb700 1 -- 172.21.15.99:0/2718941274 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68020070 con 0x5648029f8e40 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb75ffb700 1 -- 172.21.15.99:0/2718941274 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb680b1bf0 con 0x5648029f8e40 2024-05-14T17:15:16.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.768+0000 7ffb75ffb700 1 -- 172.21.15.99:0/2718941274 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680a8250 con 0x5648029f8e40 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2718941274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 msgr2=0x5648029e8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029e8980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffb680a8d20 tx=0x7ffb6800fef0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2718941274 shutdown_connections 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f9760 0x5648029fc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f3b70 0x5648029f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2718941274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f8e40 0x5648029e8980 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2718941274 >> 172.21.15.99:0/2718941274 conn(0x56480291e3f0 msgr2=0x5648029fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:16.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2718941274 shutdown_connections 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2718941274 wait complete. 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.769+0000 7ffb90c0ab80 1 -- start start 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 Processor -- start 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 0x564802a0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f8e40 0x564802a102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x564802a762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801ae710 con 0x5648029f3b70 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801afb80 con 0x5648029f9760 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801aede0 con 0x5648029f8e40 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb90c0ab80 10 client.0 _ll_register_callbacks cb 0x5648028d8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 0x564802a0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 0x564802a0fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50132/0 (socket says 172.21.15.99:50132) 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 -- 172.21.15.99:0/2273080477 learned_addr learned my addr 172.21.15.99:0/2273080477 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:16.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb84911700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x564802a762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f8e40 0x564802a102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 -- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f8e40 msgr2=0x564802a102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f8e40 0x564802a102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 -- 172.21.15.99:0/2273080477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 msgr2=0x564802a762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x564802a762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.770+0000 7ffb7ffff700 1 -- 172.21.15.99:0/2273080477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564802a767f0 con 0x5648029f3b70 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb7ffff700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 0x564802a0fd90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000c010 tx=0x7ffb70009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88119f18d983ddb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2273080477 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70038060 con 0x5648029f3b70 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2273080477 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7ffb700054c0 con 0x5648029f3b70 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb680b1bf0 con 0x5648029f3b70 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2273080477 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffb700056c0 con 0x5648029f3b70 2024-05-14T17:15:16.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 10 client.16710 fetch_fsmap learned FSMap version 40 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564802a779f0 con 0x5648029f3b70 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2273080477 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb680b1bf0 con 0x5648029f3b70 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2273080477 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7ffb70024530 con 0x5648029f3b70 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 10 client.16710 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 10 client.16710 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 20 client.16710 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a78180 0x564802a7a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 4 client.16710 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}] 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.771+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}) -- 0x564802953d80 con 0x564802a78180 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.772+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a78180 0x564802a7a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.772+0000 7ffb77fff700 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a78180 0x564802a7a5a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b81dfb9691137f server_cookie=12b57447c2f4c9a8 in_seq=0 out_seq=0 2024-05-14T17:15:16.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.772+0000 7ffb5f7fe700 10 client.16710 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.837+0000 7ffb5f7fe700 1 -- 172.21.15.99:0/2273080477 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564802953d80 con 0x564802a78180 2024-05-14T17:15:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.837+0000 7ffb5f7fe700 10 client.16710 handle_command_reply: tid=0 2024-05-14T17:15:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.837+0000 7ffb90c0ab80 1 client.16710 shutdown 2024-05-14T17:15:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.837+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 msgr2=0x564802a0fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.837+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 0x564802a0fd90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000c010 tx=0x7ffb70009cd0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 shutdown_connections 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648029f9760 0x564802a762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x564802a78180 0x564802a7a5a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb5f7fe700 0 client.16710 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648029f3b70 0x564802a0fd90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 --2- 172.21.15.99:0/2273080477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648029f8e40 0x564802a102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 >> 172.21.15.99:0/2273080477 conn(0x56480291e3f0 msgr2=0x564802a08c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:15:16.839 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 40, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 38, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "2d6e7d5d": { 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 38, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 149.7, 2024-05-14T17:15:16.840 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:16.841 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:15:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 shutdown_connections 2024-05-14T17:15:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 1 -- 172.21.15.99:0/2273080477 wait complete. 2024-05-14T17:15:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.838+0000 7ffb90c0ab80 20 client.16710 trim_cache size 0 max 16384 2024-05-14T17:15:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ffb6c0b2b20 msgr2=0x7ffb6c0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ffb6c0b2b20 0x7ffb6c0b4fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b580 tx=0x7ffb700058e0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 msgr2=0x7ffb80075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 0x7ffb80075d80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800d950 tx=0x7ffb6800dc60 comp rx=0 tx=0).stop 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 shutdown_connections 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ffb6c0b2b20 0x7ffb6c0b4fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb80108920 0x7ffb800762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb80103850 0x7ffb80075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 --2- 172.21.15.99:0/4127831734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb80104200 0x7ffb80075d80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.839+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 >> 172.21.15.99:0/4127831734 conn(0x7ffb8007a620 msgr2=0x7ffb801015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.840+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 shutdown_connections 2024-05-14T17:15:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:16.840+0000 7ffb86374700 1 -- 172.21.15.99:0/4127831734 wait complete. 2024-05-14T17:15:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:17 smithi193 ceph-mon[35962]: pgmap v366: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.4 MiB/s wr, 100 op/s 2024-05-14T17:15:17.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:17 smithi150 ceph-mon[36204]: pgmap v366: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.4 MiB/s wr, 100 op/s 2024-05-14T17:15:17.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:17 smithi099 ceph-mon[31851]: pgmap v366: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.4 MiB/s wr, 100 op/s 2024-05-14T17:15:17.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.19 MB/sec execute 449 sec latency 689.796 ms 2024-05-14T17:15:17.605 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2d6e7d5d', quiesce took 6.1 seconds. Will release after: 23.549999999999997 2024-05-14T17:15:18.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.18 MB/sec execute 450 sec latency 1689.894 ms 2024-05-14T17:15:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:19 smithi193 ceph-mon[35962]: pgmap v367: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-05-14T17:15:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:19 smithi150 ceph-mon[36204]: pgmap v367: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-05-14T17:15:19.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:19 smithi099 ceph-mon[31851]: pgmap v367: 97 pgs: 97 active+clean; 136 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.3 MiB/s wr, 97 op/s 2024-05-14T17:15:19.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.17 MB/sec execute 451 sec latency 2689.955 ms 2024-05-14T17:15:20.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.16 MB/sec execute 452 sec latency 3690.040 ms 2024-05-14T17:15:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:21 smithi193 ceph-mon[35962]: pgmap v368: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-05-14T17:15:21.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:21 smithi150 ceph-mon[36204]: pgmap v368: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-05-14T17:15:21.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:21 smithi099 ceph-mon[31851]: pgmap v368: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-05-14T17:15:21.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.15 MB/sec execute 453 sec latency 4690.131 ms 2024-05-14T17:15:22.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.14 MB/sec execute 454 sec latency 5690.184 ms 2024-05-14T17:15:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:23 smithi193 ceph-mon[35962]: pgmap v369: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-05-14T17:15:23.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:23 smithi150 ceph-mon[36204]: pgmap v369: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-05-14T17:15:23.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:23 smithi099 ceph-mon[31851]: pgmap v369: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 127 op/s 2024-05-14T17:15:23.556 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.13 MB/sec execute 455 sec latency 6690.288 ms 2024-05-14T17:15:24.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.12 MB/sec execute 456 sec latency 7690.408 ms 2024-05-14T17:15:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:25 smithi193 ceph-mon[35962]: pgmap v370: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-05-14T17:15:25.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:25 smithi150 ceph-mon[36204]: pgmap v370: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-05-14T17:15:25.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:25 smithi099 ceph-mon[31851]: pgmap v370: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-05-14T17:15:25.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.12 MB/sec execute 457 sec latency 8690.527 ms 2024-05-14T17:15:26.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:26 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:26.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:26 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:26.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:26 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:26.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.11 MB/sec execute 458 sec latency 9690.595 ms 2024-05-14T17:15:27.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:27 smithi150 ceph-mon[36204]: pgmap v371: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-14T17:15:27.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:27 smithi099 ceph-mon[31851]: pgmap v371: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-14T17:15:27.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:27 smithi193 ceph-mon[35962]: pgmap v371: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-14T17:15:27.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.10 MB/sec execute 459 sec latency 10690.681 ms 2024-05-14T17:15:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.890+0000 7f89ca299640 1 -- 172.21.15.99:0/4190505373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c4101c30 msgr2=0x7f89c41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.890+0000 7f89ca299640 1 --2- 172.21.15.99:0/4190505373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c4101c30 0x7f89c41020b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009900 tx=0x7f89b802a410 comp rx=0 tx=0).stop 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 -- 172.21.15.99:0/4190505373 shutdown_connections 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 --2- 172.21.15.99:0/4190505373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c41025f0 0x7f89c410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 --2- 172.21.15.99:0/4190505373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c4101c30 0x7f89c41020b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 --2- 172.21.15.99:0/4190505373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89c4106920 0x7f89c4106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 -- 172.21.15.99:0/4190505373 >> 172.21.15.99:0/4190505373 conn(0x7f89c4078730 msgr2=0x7f89c4078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 -- 172.21.15.99:0/4190505373 shutdown_connections 2024-05-14T17:15:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 -- 172.21.15.99:0/4190505373 wait complete. 2024-05-14T17:15:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 Processor -- start 2024-05-14T17:15:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.891+0000 7f89ca299640 1 -- start start 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89ca299640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c4101c30 0x7f89c40ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89ca299640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 0x7f89c40ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89ca299640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89c4106920 0x7f89c4100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89ca299640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4111bc0 con 0x7f89c41025f0 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89ca299640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4111a40 con 0x7f89c4101c30 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89ca299640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4111d40 con 0x7f89c4106920 2024-05-14T17:15:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 0x7f89c40ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 0x7f89c40ffc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53462/0 (socket says 172.21.15.99:53462) 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 -- 172.21.15.99:0/1080746079 learned_addr learned my addr 172.21.15.99:0/1080746079 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c8836640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89c4106920 0x7f89c4100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c4101c30 0x7f89c40ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 -- 172.21.15.99:0/1080746079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89c4106920 msgr2=0x7f89c4100140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89c4106920 0x7f89c4100140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 -- 172.21.15.99:0/1080746079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c4101c30 msgr2=0x7f89c40ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c4101c30 0x7f89c40ff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 -- 172.21.15.99:0/1080746079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c41a7610 con 0x7f89c41025f0 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c4101c30 0x7f89c40ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 0x7f89c40ffc00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f89b8005940 tx=0x7f89b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.892+0000 7f89c17fa640 1 -- 172.21.15.99:0/1080746079 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b803f030 con 0x7f89c41025f0 2024-05-14T17:15:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.893+0000 7f89c17fa640 1 -- 172.21.15.99:0/1080746079 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b8003aa0 con 0x7f89c41025f0 2024-05-14T17:15:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.893+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c41a78a0 con 0x7f89c41025f0 2024-05-14T17:15:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.893+0000 7f89c17fa640 1 -- 172.21.15.99:0/1080746079 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8035070 con 0x7f89c41025f0 2024-05-14T17:15:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.893+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c41a7d90 con 0x7f89c41025f0 2024-05-14T17:15:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 Processor -- start 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89c17fa640 1 -- 172.21.15.99:0/1080746079 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f89b8042030 con 0x7f89c41025f0 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 -- start start 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x55607037ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037f210 0x556070389590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x55607038bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560702f1c30 con 0x55607037e8f0 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5560702e17d0 con 0x556070389ad0 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560702e5810 con 0x55607037f210 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.895+0000 7f89c8836640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x55607037ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x55607037ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53478/0 (socket says 172.21.15.99:53478) 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 -- 172.21.15.99:0/1948056771 learned_addr learned my addr 172.21.15.99:0/1948056771 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037f210 0x556070389590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x55607038bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 -- 172.21.15.99:0/1948056771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037f210 msgr2=0x556070389590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037f210 0x556070389590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 -- 172.21.15.99:0/1948056771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 msgr2=0x55607038bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x55607038bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 -- 172.21.15.99:0/1948056771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606fff6210 con 0x55607037e8f0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x55607038bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c17fa640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f89ac0b2a10 0x7f89ac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c8836640 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x55607037ecd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f89bc010370 tx=0x7f89bc010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7bae0b05d362d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f89ac0b2a10 0x7f89ac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89aa7fc640 1 -- 172.21.15.99:0/1948056771 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc016070 con 0x55607037e8f0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89c17fa640 1 -- 172.21.15.99:0/1080746079 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f89b8102970 con 0x7f89c41025f0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89aa7fc640 1 -- 172.21.15.99:0/1948056771 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc004d20 con 0x55607037e8f0 2024-05-14T17:15:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.896+0000 7f89aa7fc640 1 -- 172.21.15.99:0/1948056771 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc005350 con 0x55607037e8f0 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1948056771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 msgr2=0x55607037ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f89ac0b2a10 0x7f89ac0b4ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55607037fe90 tx=0x7f89b4005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x55607037ecd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f89bc010370 tx=0x7f89bc010830 comp rx=0 tx=0).stop 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1948056771 shutdown_connections 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x55607038bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037f210 0x556070389590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1948056771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x55607037ecd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1948056771 >> 172.21.15.99:0/1948056771 conn(0x55607037ae00 msgr2=0x55607037b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1948056771 shutdown_connections 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1948056771 wait complete. 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.897+0000 7f89ccfbc740 1 -- start start 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 Processor -- start 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037e8f0 0x5560703894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x5560703867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x556070386dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55606ffc7790 con 0x55607037f210 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55607029d890 con 0x556070389ad0 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55607038c7a0 con 0x55607037e8f0 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89ccfbc740 10 client.0 _ll_register_callbacks cb 0x5560702a0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x5560703867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x5560703867b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53490/0 (socket says 172.21.15.99:53490) 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 -- 172.21.15.99:0/3804463195 learned_addr learned my addr 172.21.15.99:0/3804463195 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c3fff640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037e8f0 0x5560703894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c8836640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x556070386dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 -- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037e8f0 msgr2=0x5560703894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037e8f0 0x5560703894c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 -- 172.21.15.99:0/3804463195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 msgr2=0x556070386dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x556070386dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c37fe640 1 -- 172.21.15.99:0/3804463195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560700c2620 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.898+0000 7f89c8836640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x556070386dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89c37fe640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x5560703867b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f89b80c6ec0 tx=0x7f89b8003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ca6d80568e7dd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89a37fe640 1 -- 172.21.15.99:0/3804463195 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8049040 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89a37fe640 1 -- 172.21.15.99:0/3804463195 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f89b80c5ee0 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc004d20 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89a37fe640 1 -- 172.21.15.99:0/3804463195 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f89b80b1040 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 10 client.16722 fetch_fsmap learned FSMap version 41 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5560703981b0 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89a37fe640 1 -- 172.21.15.99:0/3804463195 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc004d20 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89a37fe640 1 -- 172.21.15.99:0/3804463195 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8342+0+0 (secure 0 0 0) 0x7f89b803bcc0 con 0x55607037f210 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 10 client.16722 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 20 client.16722 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070399610 0x55607039ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 4 client.16722 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.899+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560702f3b80 con 0x556070399610 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.900+0000 7f89c3fff640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070399610 0x55607039ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.900+0000 7f89a37fe640 10 client.16722 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.900+0000 7f89c3fff640 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070399610 0x55607039ba70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a92479d2910930 server_cookie=41d0a600cb4659f in_seq=0 out_seq=0 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89a37fe640 1 -- 172.21.15.99:0/3804463195 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560702f3b80 con 0x556070399610 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89a37fe640 10 client.16722 handle_command_reply: tid=0 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 client.16722 shutdown 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 msgr2=0x5560703867b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x5560703867b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f89b80c6ec0 tx=0x7f89b8003e80 comp rx=0 tx=0).stop 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 shutdown_connections 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070399610 0x55607039ba70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556070389ad0 0x556070386dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89a37fe640 0 client.16722 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x5560703867b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/3804463195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55607037e8f0 0x5560703894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 >> 172.21.15.99:0/3804463195 conn(0x55607037ae00 msgr2=0x55607037a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 shutdown_connections 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 1 -- 172.21.15.99:0/3804463195 wait complete. 2024-05-14T17:15:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.901+0000 7f89ccfbc740 20 client.16722 trim_cache size 0 max 16384 2024-05-14T17:15:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 Processor -- start 2024-05-14T17:15:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 -- start start 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x556070381410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037f210 0x556070388ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41b3040 con 0x55607037e8f0 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41b08e0 con 0x55607037f210 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41ae750 con 0x556070389ad0 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x556070381410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x556070381410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53502/0 (socket says 172.21.15.99:53502) 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 -- 172.21.15.99:0/461063107 learned_addr learned my addr 172.21.15.99:0/461063107 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c8836640 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c37fe640 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037f210 0x556070388ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 -- 172.21.15.99:0/461063107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 msgr2=0x5560703a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 -- 172.21.15.99:0/461063107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037f210 msgr2=0x556070388ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037f210 0x556070388ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c3fff640 1 -- 172.21.15.99:0/461063107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556070384d00 con 0x55607037e8f0 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.917+0000 7f89c37fe640 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037f210 0x556070388ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89c3fff640 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x556070381410 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55607039c2b0 tx=0x7f89b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312071c1cd97ddc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89a9ffb640 1 -- 172.21.15.99:0/461063107 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b401fd90 con 0x55607037e8f0 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89a9ffb640 1 -- 172.21.15.99:0/461063107 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b40054d0 con 0x55607037e8f0 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89a9ffb640 1 -- 172.21.15.99:0/461063107 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b402a410 con 0x55607037e8f0 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89ccfbc740 1 -- 172.21.15.99:0/461063107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 msgr2=0x556070381410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x556070381410 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55607039c2b0 tx=0x7f89b4016000 comp rx=0 tx=0).stop 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.918+0000 7f89ccfbc740 1 -- 172.21.15.99:0/461063107 shutdown_connections 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037f210 0x556070388ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/461063107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037e8f0 0x556070381410 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- 172.21.15.99:0/461063107 >> 172.21.15.99:0/461063107 conn(0x55607037ae00 msgr2=0x556070380b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- 172.21.15.99:0/461063107 shutdown_connections 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- 172.21.15.99:0/461063107 wait complete. 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- start start 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 Processor -- start 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037e8f0 0x55607038ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x55607038e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41aeda0 con 0x55607037f210 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41b0210 con 0x55607037e8f0 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41af470 con 0x556070389ad0 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89ccfbc740 10 client.0 _ll_register_callbacks cb 0x7f89b8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x55607038e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x55607038e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53506/0 (socket says 172.21.15.99:53506) 2024-05-14T17:15:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.919+0000 7f89c37fe640 1 -- 172.21.15.99:0/1538736853 learned_addr learned my addr 172.21.15.99:0/1538736853 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037e8f0 0x55607038ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c8836640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c37fe640 1 -- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 msgr2=0x5560703fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c37fe640 1 -- 172.21.15.99:0/1538736853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037e8f0 msgr2=0x55607038ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037e8f0 0x55607038ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c37fe640 1 -- 172.21.15.99:0/1538736853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560703fd480 con 0x55607037f210 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89c37fe640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x55607038e2f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f89b81006c0 tx=0x7f89b80c6110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48381744942017fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89a37fe640 1 -- 172.21.15.99:0/1538736853 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8049070 con 0x55607037f210 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89a37fe640 1 -- 172.21.15.99:0/1538736853 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f89b80c6240 con 0x55607037f210 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b40054d0 con 0x55607037f210 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89a37fe640 1 -- 172.21.15.99:0/1538736853 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f89b80ab050 con 0x55607037f210 2024-05-14T17:15:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89ccfbc740 10 client.16728 fetch_fsmap learned FSMap version 41 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5560703fd480 con 0x55607037f210 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89a37fe640 1 -- 172.21.15.99:0/1538736853 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b40054d0 con 0x55607037f210 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.920+0000 7f89a37fe640 1 -- 172.21.15.99:0/1538736853 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8342+0+0 (secure 0 0 0) 0x7f89b80c9eb0 con 0x55607037f210 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89ccfbc740 10 client.16728 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89ccfbc740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89ccfbc740 20 client.16728 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070404be0 0x556070407000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89ccfbc740 4 client.16728 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560702ed360 con 0x556070404be0 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070404be0 0x556070407000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89a37fe640 10 client.16728 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.921+0000 7f89c3fff640 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070404be0 0x556070407000 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8ca8b5d0fef90e server_cookie=ddadd0a10604256a in_seq=0 out_seq=0 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89a37fe640 1 -- 172.21.15.99:0/1538736853 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560702ed360 con 0x556070404be0 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89a37fe640 10 client.16728 handle_command_reply: tid=0 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 client.16728 shutdown 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 msgr2=0x55607038e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x55607038e2f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f89b81006c0 tx=0x7f89b80c6110 comp rx=0 tx=0).stop 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 shutdown_connections 2024-05-14T17:15:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x556070404be0 0x556070407000 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556070389ad0 0x5560703fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55607037f210 0x55607038e2f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89a37fe640 0 client.16728 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 --2- 172.21.15.99:0/1538736853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55607037e8f0 0x55607038ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 >> 172.21.15.99:0/1538736853 conn(0x55607037ae00 msgr2=0x5560703802a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 shutdown_connections 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 1 -- 172.21.15.99:0/1538736853 wait complete. 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ccfbc740 20 client.16728 trim_cache size 0 max 16384 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:15:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:15:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:15:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f89ac0b2a10 msgr2=0x7f89ac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f89ac0b2a10 0x7f89ac0b4ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55607037fe90 tx=0x7f89b4005f90 comp rx=0 tx=0).stop 2024-05-14T17:15:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 msgr2=0x7f89c40ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 0x7f89c40ffc00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f89b8005940 tx=0x7f89b8004270 comp rx=0 tx=0).stop 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 shutdown_connections 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f89ac0b2a10 0x7f89ac0b4ed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89c4106920 0x7f89c4100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89c41025f0 0x7f89c40ffc00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 --2- 172.21.15.99:0/1080746079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89c4101c30 0x7f89c40ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 >> 172.21.15.99:0/1080746079 conn(0x7f89c4078730 msgr2=0x7f89c406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 shutdown_connections 2024-05-14T17:15:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:27.956+0000 7f89ca299640 1 -- 172.21.15.99:0/1080746079 wait complete. 2024-05-14T17:15:28.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.09 MB/sec execute 460 sec latency 11690.769 ms 2024-05-14T17:15:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:29 smithi150 ceph-mon[36204]: pgmap v372: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-14T17:15:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:29 smithi099 ceph-mon[31851]: pgmap v372: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-14T17:15:29.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:29 smithi193 ceph-mon[35962]: pgmap v372: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-05-14T17:15:29.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.08 MB/sec execute 461 sec latency 12690.847 ms 2024-05-14T17:15:30.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.07 MB/sec execute 462 sec latency 13690.934 ms 2024-05-14T17:15:31.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:31 smithi150 ceph-mon[36204]: pgmap v373: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-14T17:15:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:31 smithi099 ceph-mon[31851]: pgmap v373: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-14T17:15:31.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:31 smithi193 ceph-mon[35962]: pgmap v373: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-05-14T17:15:31.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.06 MB/sec execute 463 sec latency 14691.040 ms 2024-05-14T17:15:32.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.05 MB/sec execute 464 sec latency 15691.122 ms 2024-05-14T17:15:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:33 smithi150 ceph-mon[36204]: pgmap v374: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 556 KiB/s wr, 39 op/s 2024-05-14T17:15:33.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:33 smithi099 ceph-mon[31851]: pgmap v374: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 556 KiB/s wr, 39 op/s 2024-05-14T17:15:33.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:33 smithi193 ceph-mon[35962]: pgmap v374: 97 pgs: 97 active+clean; 139 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 556 KiB/s wr, 39 op/s 2024-05-14T17:15:33.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.04 MB/sec execute 465 sec latency 16691.197 ms 2024-05-14T17:15:34.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.04 MB/sec execute 466 sec latency 17691.276 ms 2024-05-14T17:15:35.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:35 smithi150 ceph-mon[36204]: pgmap v375: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 873 KiB/s wr, 49 op/s 2024-05-14T17:15:35.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:35 smithi099 ceph-mon[31851]: pgmap v375: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 873 KiB/s wr, 49 op/s 2024-05-14T17:15:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:35 smithi193 ceph-mon[35962]: pgmap v375: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 873 KiB/s wr, 49 op/s 2024-05-14T17:15:35.557 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.03 MB/sec execute 467 sec latency 18691.400 ms 2024-05-14T17:15:36.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.02 MB/sec execute 468 sec latency 19691.455 ms 2024-05-14T17:15:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:37 smithi150 ceph-mon[36204]: pgmap v376: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 605 KiB/s wr, 34 op/s 2024-05-14T17:15:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:37 smithi099 ceph-mon[31851]: pgmap v376: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 605 KiB/s wr, 34 op/s 2024-05-14T17:15:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:37 smithi193 ceph-mon[35962]: pgmap v376: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 605 KiB/s wr, 34 op/s 2024-05-14T17:15:37.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.01 MB/sec execute 469 sec latency 20691.605 ms 2024-05-14T17:15:38.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 4.00 MB/sec execute 470 sec latency 21691.734 ms 2024-05-14T17:15:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:39 smithi150 ceph-mon[36204]: pgmap v377: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 605 KiB/s wr, 34 op/s 2024-05-14T17:15:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:39 smithi099 ceph-mon[31851]: pgmap v377: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 605 KiB/s wr, 34 op/s 2024-05-14T17:15:39.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:39 smithi193 ceph-mon[35962]: pgmap v377: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 605 KiB/s wr, 34 op/s 2024-05-14T17:15:39.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.99 MB/sec execute 471 sec latency 22691.873 ms 2024-05-14T17:15:40.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.99 MB/sec execute 472 sec latency 23691.953 ms 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 -- 172.21.15.99:0/3286740093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f964406be50 msgr2=0x7f96441054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 --2- 172.21.15.99:0/3286740093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f964406be50 0x7f96441054f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f963c009900 tx=0x7f963c02a410 comp rx=0 tx=0).stop 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 -- 172.21.15.99:0/3286740093 shutdown_connections 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 --2- 172.21.15.99:0/3286740093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f964406be50 0x7f96441054f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 --2- 172.21.15.99:0/3286740093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9644107e00 0x7f964406b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 --2- 172.21.15.99:0/3286740093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644107990 0x7f964406b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 -- 172.21.15.99:0/3286740093 >> 172.21.15.99:0/3286740093 conn(0x7f9644078730 msgr2=0x7f9644078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 -- 172.21.15.99:0/3286740093 shutdown_connections 2024-05-14T17:15:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 -- 172.21.15.99:0/3286740093 wait complete. 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.054+0000 7f964b420640 1 Processor -- start 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 -- start start 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9644107990 0x7f964419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644107e00 0x7f964419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 0x7f96441a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96441192f0 con 0x7f96441a4010 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9644119170 con 0x7f9644107e00 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f964b420640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9644119470 con 0x7f9644107990 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 0x7f96441a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 0x7f96441a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34814/0 (socket says 172.21.15.99:34814) 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 -- 172.21.15.99:0/2246738828 learned_addr learned my addr 172.21.15.99:0/2246738828 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96491bc640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9644107990 0x7f964419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96489bb640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644107e00 0x7f964419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 -- 172.21.15.99:0/2246738828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9644107990 msgr2=0x7f964419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9644107990 0x7f964419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 -- 172.21.15.99:0/2246738828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644107e00 msgr2=0x7f964419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644107e00 0x7f964419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 -- 172.21.15.99:0/2246738828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a4b00 con 0x7f96441a4010 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.055+0000 7f96499bd640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 0x7f96441a44c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f963c004770 tx=0x7f963c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.056+0000 7f963a7fc640 1 -- 172.21.15.99:0/2246738828 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c03f030 con 0x7f96441a4010 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.056+0000 7f963a7fc640 1 -- 172.21.15.99:0/2246738828 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963c003aa0 con 0x7f96441a4010 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.056+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a4d90 con 0x7f96441a4010 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.056+0000 7f963a7fc640 1 -- 172.21.15.99:0/2246738828 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c035070 con 0x7f96441a4010 2024-05-14T17:15:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.056+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441ac550 con 0x7f96441a4010 2024-05-14T17:15:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.058+0000 7f963a7fc640 1 -- 172.21.15.99:0/2246738828 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f963c042030 con 0x7f96441a4010 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.058+0000 7f964e215740 1 Processor -- start 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.058+0000 7f964e215740 1 -- start start 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f964e215740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee20cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f964e215740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f964e215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f964e215740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56418ed93c30 con 0x56418ee208f0 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f964e215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56418ed837d0 con 0x56418ee2bad0 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f964e215740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56418ed87810 con 0x56418ee21210 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee20cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee20cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34830/0 (socket says 172.21.15.99:34830) 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 -- 172.21.15.99:0/2211450835 learned_addr learned my addr 172.21.15.99:0/2211450835 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96491bc640 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 -- 172.21.15.99:0/2211450835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 msgr2=0x56418ee2b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 -- 172.21.15.99:0/2211450835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 msgr2=0x56418ee2dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 -- 172.21.15.99:0/2211450835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418ea98210 con 0x56418ee208f0 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f963a7fc640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f96300b2a10 0x7f96300b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96489bb640 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee20cd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f96340058c0 tx=0x7f9634011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dd9616ebdc854c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f96491bc640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f96300b2a10 0x7f96300b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f963a7fc640 1 -- 172.21.15.99:0/2246738828 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f963c102a30 con 0x7f96441a4010 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.059+0000 7f962f7fe640 1 -- 172.21.15.99:0/2211450835 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963401a400 con 0x56418ee208f0 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f962f7fe640 1 -- 172.21.15.99:0/2211450835 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96340128a0 con 0x56418ee208f0 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f962f7fe640 1 -- 172.21.15.99:0/2211450835 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963400f740 con 0x56418ee208f0 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 -- 172.21.15.99:0/2211450835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 msgr2=0x56418ee20cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f96491bc640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f96300b2a10 0x7f96300b4ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x56418ee21e90 tx=0x7f9640009680 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee20cd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f96340058c0 tx=0x7f9634011750 comp rx=0 tx=0).stop 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 -- 172.21.15.99:0/2211450835 shutdown_connections 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee2dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 --2- 172.21.15.99:0/2211450835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee20cd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 -- 172.21.15.99:0/2211450835 >> 172.21.15.99:0/2211450835 conn(0x56418ee1ce00 msgr2=0x56418ee1d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 -- 172.21.15.99:0/2211450835 shutdown_connections 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.060+0000 7f964e215740 1 -- 172.21.15.99:0/2211450835 wait complete. 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 -- start start 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 Processor -- start 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee2b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee28bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56418ea69790 con 0x56418ee21210 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56418ed3f890 con 0x56418ee2bad0 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56418ee2e7a0 con 0x56418ee208f0 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f964e215740 10 client.0 _ll_register_callbacks cb 0x56418ed42d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f96489bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f96489bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee286b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34836/0 (socket says 172.21.15.99:34836) 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f96489bb640 1 -- 172.21.15.99:0/354374175 learned_addr learned my addr 172.21.15.99:0/354374175 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f96499bd640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee28bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.061+0000 7f96491bc640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee2b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96489bb640 1 -- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 msgr2=0x56418ee2b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96489bb640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee2b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96489bb640 1 -- 172.21.15.99:0/354374175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 msgr2=0x56418ee28bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96489bb640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee28bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96489bb640 1 -- 172.21.15.99:0/354374175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418eb64620 con 0x56418ee21210 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96491bc640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee2b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f96489bb640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee286b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f963400f420 tx=0x7f963400ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121991cdd100e3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f962cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f962cff9640 1 -- 172.21.15.99:0/354374175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963402a060 con 0x56418ee21210 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96340128a0 con 0x56418ee21210 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f962cff9640 1 -- 172.21.15.99:0/354374175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f9634012a50 con 0x56418ee21210 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f962cff9640 1 -- 172.21.15.99:0/354374175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9634012c50 con 0x56418ee21210 2024-05-14T17:15:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f964e215740 10 client.16740 fetch_fsmap learned FSMap version 41 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56418ee3a200 con 0x56418ee21210 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.062+0000 7f962cff9640 1 -- 172.21.15.99:0/354374175 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96340128a0 con 0x56418ee21210 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f962cff9640 1 -- 172.21.15.99:0/354374175 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8342+0+0 (secure 0 0 0) 0x7f9634004a60 con 0x56418ee21210 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f964e215740 10 client.16740 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f964e215740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f964e215740 20 client.16740 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f964e215740 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418ee3b660 0x56418ee3da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f964e215740 4 client.16740 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418ed95b80 con 0x56418ee3b660 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f96491bc640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418ee3b660 0x56418ee3da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f96491bc640 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418ee3b660 0x56418ee3da50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff4b0bd932ed0f4 server_cookie=104b0f7e1ea046c9 in_seq=0 out_seq=0 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.063+0000 7f962cff9640 10 client.16740 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f962cff9640 1 -- 172.21.15.99:0/354374175 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56418ed95b80 con 0x56418ee3b660 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f962cff9640 10 client.16740 handle_command_reply: tid=0 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 client.16740 shutdown 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 msgr2=0x56418ee286b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee286b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f963400f420 tx=0x7f963400ba70 comp rx=0 tx=0).stop 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 shutdown_connections 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418ee3b660 0x56418ee3da50 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee28bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee286b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 --2- 172.21.15.99:0/354374175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee2b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f962cff9640 0 client.16740 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 >> 172.21.15.99:0/354374175 conn(0x56418ee1ce00 msgr2=0x56418ee1c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 shutdown_connections 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 1 -- 172.21.15.99:0/354374175 wait complete. 2024-05-14T17:15:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.064+0000 7f964e215740 20 client.16740 trim_cache size 0 max 16384 2024-05-14T17:15:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 Processor -- start 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 -- start start 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee43050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b7450 con 0x56418ee208f0 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b4cf0 con 0x56418ee2bad0 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f964e215740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b2b60 con 0x56418ee21210 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f96491bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f96491bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee27920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34846/0 (socket says 172.21.15.99:34846) 2024-05-14T17:15:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.080+0000 7f96491bc640 1 -- 172.21.15.99:0/2410380468 learned_addr learned my addr 172.21.15.99:0/2410380468 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f96491bc640 1 -- 172.21.15.99:0/2410380468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 msgr2=0x56418ee2aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f96491bc640 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f96491bc640 1 -- 172.21.15.99:0/2410380468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 msgr2=0x56418ee43050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f96491bc640 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee43050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f96491bc640 1 -- 172.21.15.99:0/2410380468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418ee1cc30 con 0x56418ee208f0 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f96491bc640 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee27920 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56418ee3dff0 tx=0x7f9640016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abed10a4022e4c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f962effd640 1 -- 172.21.15.99:0/2410380468 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640004d20 con 0x56418ee208f0 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f962effd640 1 -- 172.21.15.99:0/2410380468 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640029860 con 0x56418ee208f0 2024-05-14T17:15:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.081+0000 7f962effd640 1 -- 172.21.15.99:0/2410380468 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96400057a0 con 0x56418ee208f0 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 -- 172.21.15.99:0/2410380468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 msgr2=0x56418ee27920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee27920 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56418ee3dff0 tx=0x7f9640016000 comp rx=0 tx=0).stop 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 -- 172.21.15.99:0/2410380468 shutdown_connections 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee43050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee21210 0x56418ee2aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 --2- 172.21.15.99:0/2410380468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee208f0 0x56418ee27920 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 -- 172.21.15.99:0/2410380468 >> 172.21.15.99:0/2410380468 conn(0x56418ee1ce00 msgr2=0x56418ee22b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 -- 172.21.15.99:0/2410380468 shutdown_connections 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 -- 172.21.15.99:0/2410380468 wait complete. 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 -- start start 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.082+0000 7f964e215740 1 Processor -- start 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee25450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee25990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee25ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b31b0 con 0x56418ee21210 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b4620 con 0x56418ee2bad0 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b3880 con 0x56418ee208f0 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 10 client.0 _ll_register_callbacks cb 0x7f963400b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee25990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee25990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34860/0 (socket says 172.21.15.99:34860) 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 -- 172.21.15.99:0/3917217265 learned_addr learned my addr 172.21.15.99:0/3917217265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96491bc640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee25450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96499bd640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee25ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 -- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 msgr2=0x56418ee25450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee25450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 -- 172.21.15.99:0/3917217265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 msgr2=0x56418ee25ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee25ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 -- 172.21.15.99:0/3917217265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418ee21bb0 con 0x56418ee21210 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96499bd640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee25ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f96489bb640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee25990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9634005870 tx=0x7f963400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef1525a8ee6d11c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f962cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f962cff9640 1 -- 172.21.15.99:0/3917217265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963401e020 con 0x56418ee21210 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f962cff9640 1 -- 172.21.15.99:0/3917217265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f9634022040 con 0x56418ee21210 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9640029860 con 0x56418ee21210 2024-05-14T17:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f962cff9640 1 -- 172.21.15.99:0/3917217265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f963403d050 con 0x56418ee21210 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 10 client.16746 fetch_fsmap learned FSMap version 41 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.083+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56418ee2e290 con 0x56418ee21210 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f962cff9640 1 -- 172.21.15.99:0/3917217265 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9640029860 con 0x56418ee21210 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f962cff9640 1 -- 172.21.15.99:0/3917217265 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8342+0+0 (secure 0 0 0) 0x7f9634004710 con 0x56418ee21210 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f964e215740 10 client.16746 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f964e215740 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f964e215740 20 client.16746 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f964e215740 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418eea6180 0x56418eea85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f964e215740 4 client.16746 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56418ee18630 con 0x56418eea6180 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.084+0000 7f96491bc640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418eea6180 0x56418eea85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.085+0000 7f96491bc640 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418eea6180 0x56418eea85a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b0fb653dce5a51 server_cookie=4816000d98643cf1 in_seq=0 out_seq=0 2024-05-14T17:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.085+0000 7f962cff9640 10 client.16746 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f962cff9640 1 -- 172.21.15.99:0/3917217265 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56418ee18630 con 0x56418eea6180 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f962cff9640 10 client.16746 handle_command_reply: tid=0 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 client.16746 shutdown 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 msgr2=0x56418ee25990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee25990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9634005870 tx=0x7f963400b650 comp rx=0 tx=0).stop 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 shutdown_connections 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56418eea6180 0x56418eea85a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56418ee2bad0 0x56418ee25ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f962cff9640 0 client.16746 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56418ee21210 0x56418ee25990 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 --2- 172.21.15.99:0/3917217265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56418ee208f0 0x56418ee25450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 >> 172.21.15.99:0/3917217265 conn(0x56418ee1ce00 msgr2=0x56418ee222a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 shutdown_connections 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 1 -- 172.21.15.99:0/3917217265 wait complete. 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.112+0000 7f964e215740 20 client.16746 trim_cache size 0 max 16384 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:15:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:15:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f96300b2a10 msgr2=0x7f96300b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f96300b2a10 0x7f96300b4ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x56418ee21e90 tx=0x7f9640009680 comp rx=0 tx=0).stop 2024-05-14T17:15:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 msgr2=0x7f96441a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 0x7f96441a44c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f963c004770 tx=0x7f963c004270 comp rx=0 tx=0).stop 2024-05-14T17:15:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 shutdown_connections 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f96300b2a10 0x7f96300b4ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96441a4010 0x7f96441a44c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644107e00 0x7f964419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 --2- 172.21.15.99:0/2246738828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9644107990 0x7f964419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 >> 172.21.15.99:0/2246738828 conn(0x7f9644078730 msgr2=0x7f96440ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 shutdown_connections 2024-05-14T17:15:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:41.113+0000 7f964b420640 1 -- 172.21.15.99:0/2246738828 wait complete. 2024-05-14T17:15:41.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:15:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:41.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:41 smithi150 ceph-mon[36204]: pgmap v378: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 762 KiB/s wr, 48 op/s 2024-05-14T17:15:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:41 smithi099 ceph-mon[31851]: pgmap v378: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 762 KiB/s wr, 48 op/s 2024-05-14T17:15:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:41 smithi193 ceph-mon[35962]: pgmap v378: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 762 KiB/s wr, 48 op/s 2024-05-14T17:15:41.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.98 MB/sec execute 473 sec latency 24692.026 ms 2024-05-14T17:15:42.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.97 MB/sec execute 474 sec latency 25692.161 ms 2024-05-14T17:15:43.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:43 smithi099 ceph-mon[31851]: pgmap v379: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 474 KiB/s wr, 23 op/s 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 -- 172.21.15.99:0/4189662164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 msgr2=0x7f819c1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 --2- 172.21.15.99:0/4189662164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c1028b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8190009990 tx=0x7f8190009ca0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 -- 172.21.15.99:0/4189662164 shutdown_connections 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 --2- 172.21.15.99:0/4189662164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f819c102df0 0x7f819c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 --2- 172.21.15.99:0/4189662164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f819c108510 0x7f819c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 --2- 172.21.15.99:0/4189662164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c1028b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 -- 172.21.15.99:0/4189662164 >> 172.21.15.99:0/4189662164 conn(0x7f819c0793b0 msgr2=0x7f819c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:43.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 -- 172.21.15.99:0/4189662164 shutdown_connections 2024-05-14T17:15:43.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.275+0000 7f81a0849700 1 -- 172.21.15.99:0/4189662164 wait complete. 2024-05-14T17:15:43.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 Processor -- start 2024-05-14T17:15:43.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 -- start start 2024-05-14T17:15:43.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f819c102df0 0x7f819c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f819c108510 0x7f819c1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1125e0 con 0x7f819c102430 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f819c112460 con 0x7f819c102df0 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f81a0849700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f819c112760 con 0x7f819c108510 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f819a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.276+0000 7f819a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34882/0 (socket says 172.21.15.99:34882) 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 -- 172.21.15.99:0/633805868 learned_addr learned my addr 172.21.15.99:0/633805868 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f8199d9b700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f819c102df0 0x7f819c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819ad9d700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f819c108510 0x7f819c1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 -- 172.21.15.99:0/633805868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f819c108510 msgr2=0x7f819c1a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f819c108510 0x7f819c1a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 -- 172.21.15.99:0/633805868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f819c102df0 msgr2=0x7f819c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f819c102df0 0x7f819c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 -- 172.21.15.99:0/633805868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c1a6000 con 0x7f819c102430 2024-05-14T17:15:43.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f819a59c700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c19aa40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f818c00d950 tx=0x7f818c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:43.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c016070 con 0x7f819c102430 2024-05-14T17:15:43.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818c010460 con 0x7f819c102430 2024-05-14T17:15:43.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c1a6230 con 0x7f819c102430 2024-05-14T17:15:43.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c00f620 con 0x7f819c102430 2024-05-14T17:15:43.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.277+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a6740 con 0x7f819c102430 2024-05-14T17:15:43.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.278+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c06a230 con 0x7f819c102430 2024-05-14T17:15:43.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.279+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f818c019070 con 0x7f819c102430 2024-05-14T17:15:43.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.280+0000 7f818b7fe700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81840b29f0 0x7f81840b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.280+0000 7f8199d9b700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81840b29f0 0x7f81840b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.280+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f818c0e2210 con 0x7f819c102430 2024-05-14T17:15:43.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.281+0000 7f8199d9b700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81840b29f0 0x7f81840b4eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8190006010 tx=0x7f819001a040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:43.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.285+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f818c0a9bf0 con 0x7f819c102430 2024-05-14T17:15:43.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:43 smithi150 ceph-mon[36204]: pgmap v379: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 474 KiB/s wr, 23 op/s 2024-05-14T17:15:43.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.532+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f819c02e680 con 0x7f819c102430 2024-05-14T17:15:43.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.533+0000 7f818b7fe700 1 -- 172.21.15.99:0/633805868 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11874 (secure 0 0 0) 0x7f818c0a94f0 con 0x7f819c102430 2024-05-14T17:15:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:15:43.532 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:15:25.006932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:15:43.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81840b29f0 msgr2=0x7f81840b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81840b29f0 0x7f81840b4eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8190006010 tx=0x7f819001a040 comp rx=0 tx=0).stop 2024-05-14T17:15:43.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 msgr2=0x7f819c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c19aa40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f818c00d950 tx=0x7f818c00dd10 comp rx=0 tx=0).stop 2024-05-14T17:15:43.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 shutdown_connections 2024-05-14T17:15:43.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81840b29f0 0x7f81840b4eb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f819c108510 0x7f819c1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f819c102430 0x7f819c19aa40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 --2- 172.21.15.99:0/633805868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f819c102df0 0x7f819c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 >> 172.21.15.99:0/633805868 conn(0x7f819c0793b0 msgr2=0x7f819c1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.535+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 shutdown_connections 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:43.536+0000 7f81a0849700 1 -- 172.21.15.99:0/633805868 wait complete. 2024-05-14T17:15:43.536 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 41 2024-05-14T17:15:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:43 smithi193 ceph-mon[35962]: pgmap v379: 97 pgs: 97 active+clean; 140 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 474 KiB/s wr, 23 op/s 2024-05-14T17:15:43.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.96 MB/sec execute 475 sec latency 26692.207 ms 2024-05-14T17:15:44.186 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 2d6e7d5d --release --await' 2024-05-14T17:15:44.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 2d6e7d5d --release --await 2024-05-14T17:15:44.439 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:44.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:44 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/633805868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:44.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:44 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/633805868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:44.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:44 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/633805868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:44.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.95 MB/sec execute 476 sec latency 27692.258 ms 2024-05-14T17:15:45.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:45 smithi193 ceph-mon[35962]: pgmap v380: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 758 KiB/s wr, 43 op/s 2024-05-14T17:15:45.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:45 smithi150 ceph-mon[36204]: pgmap v380: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 758 KiB/s wr, 43 op/s 2024-05-14T17:15:45.491 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:45 smithi099 ceph-mon[31851]: pgmap v380: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 758 KiB/s wr, 43 op/s 2024-05-14T17:15:45.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 484983 3.94 MB/sec execute 477 sec latency 28692.308 ms 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 -- 172.21.15.99:0/1969794109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 msgr2=0x7f8080108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 --2- 172.21.15.99:0/1969794109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f8080108900 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8070009990 tx=0x7f8070009ca0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 -- 172.21.15.99:0/1969794109 shutdown_connections 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 --2- 172.21.15.99:0/1969794109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080102e00 0x7f808010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 --2- 172.21.15.99:0/1969794109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f8080108900 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 --2- 172.21.15.99:0/1969794109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8080102440 0x7f80801028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 -- 172.21.15.99:0/1969794109 >> 172.21.15.99:0/1969794109 conn(0x7f80800793f0 msgr2=0x7f8080079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 -- 172.21.15.99:0/1969794109 shutdown_connections 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 -- 172.21.15.99:0/1969794109 wait complete. 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 Processor -- start 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.022+0000 7f8086165700 1 -- start start 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080102440 0x7f808019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8080102e00 0x7f808019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f80801a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80801125f0 con 0x7f8080108520 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8080112470 con 0x7f8080102440 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8080112770 con 0x7f8080102e00 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f80801a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f80801a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43722/0 (socket says 172.21.15.99:43722) 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 -- 172.21.15.99:0/3887694651 learned_addr learned my addr 172.21.15.99:0/3887694651 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807f7fe700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080102440 0x7f808019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8077fff700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8080102e00 0x7f808019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 -- 172.21.15.99:0/3887694651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8080102e00 msgr2=0x7f808019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8080102e00 0x7f808019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 -- 172.21.15.99:0/3887694651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080102440 msgr2=0x7f808019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080102440 0x7f808019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 -- 172.21.15.99:0/3887694651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80801a0e00 con 0x7f8080108520 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807ffff700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f80801a06c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8078007ee0 tx=0x7f807800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807d7fa700 1 -- 172.21.15.99:0/3887694651 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078004bb0 con 0x7f8080108520 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807d7fa700 1 -- 172.21.15.99:0/3887694651 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8078004d50 con 0x7f8080108520 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a10f0 con 0x7f8080108520 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.023+0000 7f807d7fa700 1 -- 172.21.15.99:0/3887694651 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078005920 con 0x7f8080108520 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.024+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a8970 con 0x7f8080108520 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.026+0000 7f807d7fa700 1 -- 172.21.15.99:0/3887694651 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f807801a020 con 0x7f8080108520 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.026+0000 7f807d7fa700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f806c0b29f0 0x7f806c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f807f7fe700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f806c0b29f0 0x7f806c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f807d7fa700 1 -- 172.21.15.99:0/3887694651 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f80780e1bc0 con 0x7f8080108520 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f807f7fe700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f806c0b29f0 0x7f806c0b4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f807000b5c0 tx=0x7f8070005960 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 Processor -- start 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 -- start start 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b44800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b3fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff9a9de70 con 0x558ff9b44d40 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ff9a8da10 con 0x558ff9b402c0 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f80909fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ff9a91a50 con 0x558ff9b44420 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f8077fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b3fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.027+0000 7f8077fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b3fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43736/0 (socket says 172.21.15.99:43736) 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 -- 172.21.15.99:0/450228583 learned_addr learned my addr 172.21.15.99:0/450228583 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f807ffff700 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f807f7fe700 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b44800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 -- 172.21.15.99:0/450228583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 msgr2=0x558ff9b44800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b44800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 -- 172.21.15.99:0/450228583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 msgr2=0x558ff9b40720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b40720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 -- 172.21.15.99:0/450228583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff9a370b0 con 0x558ff9b44d40 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8077fff700 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b3fca0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f806800b700 tx=0x7f806800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9c033706bf7670 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8075ffb700 1 -- 172.21.15.99:0/450228583 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068004a30 con 0x558ff9b44d40 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8075ffb700 1 -- 172.21.15.99:0/450228583 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8068015070 con 0x558ff9b44d40 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.028+0000 7f8075ffb700 1 -- 172.21.15.99:0/450228583 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068003a10 con 0x558ff9b44d40 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 -- 172.21.15.99:0/450228583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 msgr2=0x558ff9b3fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b3fca0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f806800b700 tx=0x7f806800ba10 comp rx=0 tx=0).stop 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 -- 172.21.15.99:0/450228583 shutdown_connections 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b40720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b44800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 --2- 172.21.15.99:0/450228583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b3fca0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 -- 172.21.15.99:0/450228583 >> 172.21.15.99:0/450228583 conn(0x558ff9a6ceb0 msgr2=0x558ff9b3a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 -- 172.21.15.99:0/450228583 shutdown_connections 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 -- 172.21.15.99:0/450228583 wait complete. 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.029+0000 7f80909fbb80 1 -- start start 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 Processor -- start 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b4ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 0x558ff9b48e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b49370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff99940a0 con 0x558ff9b44420 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558ff99f5ea0 con 0x558ff9b402c0 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ff9b4c880 con 0x558ff9b44d40 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f80909fbb80 10 client.0 _ll_register_callbacks cb 0x558ff9a28a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 0x558ff9b48e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 0x558ff9b48e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43740/0 (socket says 172.21.15.99:43740) 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 -- 172.21.15.99:0/1954562775 learned_addr learned my addr 172.21.15.99:0/1954562775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f807ffff700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b49370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f807f7fe700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b4ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 -- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 msgr2=0x558ff9b49370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b49370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 -- 172.21.15.99:0/1954562775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 msgr2=0x558ff9b4ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b4ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 -- 172.21.15.99:0/1954562775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff99dd9d0 con 0x558ff9b44420 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f8077fff700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 0x558ff9b48e30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f806800c010 tx=0x7f8068004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86827e1fa87eddb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.030+0000 7f805effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f805effd700 1 -- 172.21.15.99:0/1954562775 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068028060 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f805effd700 1 -- 172.21.15.99:0/1954562775 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f8068005210 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8068015070 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f805effd700 1 -- 172.21.15.99:0/1954562775 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8068005410 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 10 client.16764 fetch_fsmap learned FSMap version 42 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558ff9a47860 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f805effd700 1 -- 172.21.15.99:0/1954562775 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8068015070 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f805effd700 1 -- 172.21.15.99:0/1954562775 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f80680114d0 con 0x558ff9b44420 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 10 client.16764 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 10 client.16764 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 20 client.16764 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9b59230 0x558ff9b5b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 4 client.16764 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff9a9fdc0 con 0x558ff9b59230 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.031+0000 7f807f7fe700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9b59230 0x558ff9b5b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.032+0000 7f805effd700 10 client.16764 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.032+0000 7f807f7fe700 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9b59230 0x558ff9b5b620 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e605361d2850a34 server_cookie=f43ce7f3c450b532 in_seq=0 out_seq=0 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f805effd700 1 -- 172.21.15.99:0/1954562775 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ff9a9fdc0 con 0x558ff9b59230 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f805effd700 10 client.16764 handle_command_reply: tid=0 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 client.16764 shutdown 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 msgr2=0x558ff9b48e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 0x558ff9b48e30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f806800c010 tx=0x7f8068004a10 comp rx=0 tx=0).stop 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 shutdown_connections 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b49370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9b59230 0x558ff9b5b620 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 --2- 172.21.15.99:0/1954562775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44420 0x558ff9b48e30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f805effd700 0 client.16764 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 >> 172.21.15.99:0/1954562775 conn(0x558ff9a6ceb0 msgr2=0x558ff9b34fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 shutdown_connections 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 1 -- 172.21.15.99:0/1954562775 wait complete. 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.033+0000 7f80909fbb80 20 client.16764 trim_cache size 0 max 16384 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 Processor -- start 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 -- start start 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 0x558ff9b350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b44420 0x558ff9b3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b46200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80801b2940 con 0x558ff9b402c0 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80801b01d0 con 0x558ff9b44420 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f80909fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ae040 con 0x558ff9b44d40 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 0x558ff9b350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 0x558ff9b350d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43752/0 (socket says 172.21.15.99:43752) 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 -- 172.21.15.99:0/2190891601 learned_addr learned my addr 172.21.15.99:0/2190891601 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807ffff700 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b46200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 -- 172.21.15.99:0/2190891601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 msgr2=0x558ff9b46200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b46200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 -- 172.21.15.99:0/2190891601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b44420 msgr2=0x558ff9b3f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b44420 0x558ff9b3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 -- 172.21.15.99:0/2190891601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff9b47df0 con 0x558ff9b402c0 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.053+0000 7f807f7fe700 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 0x558ff9b350d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8070009990 tx=0x7f8070024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7832a06e26bcaefb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80757fa700 1 -- 172.21.15.99:0/2190891601 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070024a60 con 0x558ff9b402c0 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80757fa700 1 -- 172.21.15.99:0/2190891601 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8070039070 con 0x558ff9b402c0 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80757fa700 1 -- 172.21.15.99:0/2190891601 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807000f460 con 0x558ff9b402c0 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 -- 172.21.15.99:0/2190891601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 msgr2=0x558ff9b350d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 0x558ff9b350d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8070009990 tx=0x7f8070024000 comp rx=0 tx=0).stop 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 -- 172.21.15.99:0/2190891601 shutdown_connections 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44d40 0x558ff9b46200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b402c0 0x558ff9b350d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 --2- 172.21.15.99:0/2190891601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b44420 0x558ff9b3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 -- 172.21.15.99:0/2190891601 >> 172.21.15.99:0/2190891601 conn(0x558ff9a6ceb0 msgr2=0x558ff9b43970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 -- 172.21.15.99:0/2190891601 shutdown_connections 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.054+0000 7f80909fbb80 1 -- 172.21.15.99:0/2190891601 wait complete. 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 -- start start 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 Processor -- start 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b5e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b5cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b5d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80801ae690 con 0x558ff9b44d40 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80801afb00 con 0x558ff9b402c0 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f80801aed60 con 0x558ff9b44420 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f80909fbb80 10 client.0 _ll_register_callbacks cb 0x558ff9a28a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f807ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b5d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f807ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b5d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43768/0 (socket says 172.21.15.99:43768) 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f807ffff700 1 -- 172.21.15.99:0/3343173632 learned_addr learned my addr 172.21.15.99:0/3343173632 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f807f7fe700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b5e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.055+0000 7f8077fff700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b5cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f807ffff700 1 -- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 msgr2=0x558ff9b5cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f807ffff700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b5cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f807ffff700 1 -- 172.21.15.99:0/3343173632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 msgr2=0x558ff9b5e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f807ffff700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b5e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f807ffff700 1 -- 172.21.15.99:0/3343173632 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff9b5d9f0 con 0x558ff9b44d40 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f807ffff700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b5d1b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f80 tx=0x7f8078005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2359e4053a0125 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f805effd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f805effd700 1 -- 172.21.15.99:0/3343173632 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80780a89b0 con 0x558ff9b44d40 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f805effd700 1 -- 172.21.15.99:0/3343173632 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f80780a5490 con 0x558ff9b44d40 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8070039070 con 0x558ff9b44d40 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f805effd700 1 -- 172.21.15.99:0/3343173632 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f80780a95a0 con 0x558ff9b44d40 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f80909fbb80 10 client.16770 fetch_fsmap learned FSMap version 42 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558ff9bc48d0 con 0x558ff9b44d40 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.056+0000 7f805effd700 1 -- 172.21.15.99:0/3343173632 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8070039070 con 0x558ff9b44d40 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f805effd700 1 -- 172.21.15.99:0/3343173632 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f80780bf070 con 0x558ff9b44d40 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f80909fbb80 10 client.16770 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f80909fbb80 10 client.16770 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f80909fbb80 20 client.16770 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f80909fbb80 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9bc4f80 0x558ff9bc73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f80909fbb80 4 client.16770 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2d6e7d5d", "release": true, "await": true}] 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2d6e7d5d", "release": true, "await": true}) -- 0x558ff9a995a0 con 0x558ff9bc4f80 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f807f7fe700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9bc4f80 0x558ff9bc73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f805effd700 10 client.16770 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.057+0000 7f807f7fe700 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9bc4f80 0x558ff9bc73a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568d40dccd073643 server_cookie=8f62eb591108f159 in_seq=0 out_seq=0 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f805effd700 1 -- 172.21.15.99:0/3343173632 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558ff9a995a0 con 0x558ff9bc4f80 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f805effd700 10 client.16770 handle_command_reply: tid=0 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 client.16770 shutdown 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 msgr2=0x558ff9b5d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b5d1b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f80 tx=0x7f8078005280 comp rx=0 tx=0).stop 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 shutdown_connections 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x558ff9bc4f80 0x558ff9bc73a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f805effd700 0 client.16770 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff9b44d40 0x558ff9b5d1b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff9b402c0 0x558ff9b5e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 --2- 172.21.15.99:0/3343173632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff9b44420 0x558ff9b5cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 >> 172.21.15.99:0/3343173632 conn(0x558ff9a6ceb0 msgr2=0x558ff9b43970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 shutdown_connections 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 1 -- 172.21.15.99:0/3343173632 wait complete. 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.064+0000 7f80909fbb80 20 client.16770 trim_cache size 0 max 16384 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 42, 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 40, 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "2d6e7d5d": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 40, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 149.7, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f806c0b29f0 msgr2=0x7f806c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f806c0b29f0 0x7f806c0b4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f807000b5c0 tx=0x7f8070005960 comp rx=0 tx=0).stop 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 msgr2=0x7f80801a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f80801a06c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8078007ee0 tx=0x7f807800d3b0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 shutdown_connections 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f806c0b29f0 0x7f806c0b4eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8080108520 0x7f80801a06c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080102440 0x7f808019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 --2- 172.21.15.99:0/3887694651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8080102e00 0x7f808019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 >> 172.21.15.99:0/3887694651 conn(0x7f80800793f0 msgr2=0x7f808006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 shutdown_connections 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:46.065+0000 7f8086165700 1 -- 172.21.15.99:0/3887694651 wait complete. 2024-05-14T17:15:46.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:46 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:46.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:46 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:46.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:46 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:46.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 486163 3.95 MB/sec execute 478 sec latency 29200.486 ms 2024-05-14T17:15:46.936 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2d6e7d5d', total seconds elapsed: 35.5 2024-05-14T17:15:46.936 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.65 2024-05-14T17:15:46.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:15:47.184 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:47.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:47 smithi099 ceph-mon[31851]: pgmap v381: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 440 KiB/s wr, 33 op/s 2024-05-14T17:15:47.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:47 smithi150 ceph-mon[36204]: pgmap v381: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 440 KiB/s wr, 33 op/s 2024-05-14T17:15:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:47 smithi193 ceph-mon[35962]: pgmap v381: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 440 KiB/s wr, 33 op/s 2024-05-14T17:15:47.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 487992 3.96 MB/sec execute 479 sec latency 23.608 ms 2024-05-14T17:15:48.559 INFO:tasks.workunit.client.0.smithi099.stdout: 1 488453 3.96 MB/sec execute 480 sec latency 623.650 ms 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/1759593803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781088f0 msgr2=0x7fdc78110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/1759593803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781088f0 0x7fdc78110470 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b0d0 tx=0x7fdc7000b3e0 comp rx=0 tx=0).stop 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/1759593803 shutdown_connections 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/1759593803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781088f0 0x7fdc78110470 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/1759593803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781037f0 0x7fdc78103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/1759593803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 0x7fdc781081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/1759593803 >> 172.21.15.99:0/1759593803 conn(0x7fdc7807a6a0 msgr2=0x7fdc7807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/1759593803 shutdown_connections 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/1759593803 wait complete. 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 Processor -- start 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.841+0000 7fdc7f0a5700 1 -- start start 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 0x7fdc7819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 0x7fdc7819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781088f0 0x7fdc78199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc781136e0 con 0x7fdc781037f0 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78113560 con 0x7fdc781041a0 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78113860 con 0x7fdc781088f0 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 0x7fdc7819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 0x7fdc7819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43786/0 (socket says 172.21.15.99:43786) 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 -- 172.21.15.99:0/4063351749 learned_addr learned my addr 172.21.15.99:0/4063351749 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc77fff700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 0x7fdc7819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7d642700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781088f0 0x7fdc78199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 -- 172.21.15.99:0/4063351749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781088f0 msgr2=0x7fdc78199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781088f0 0x7fdc78199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 -- 172.21.15.99:0/4063351749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 msgr2=0x7fdc7819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 0x7fdc7819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 -- 172.21.15.99:0/4063351749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7819a0c0 con 0x7fdc781037f0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc77fff700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 0x7fdc7819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7d642700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781088f0 0x7fdc78199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7ce41700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 0x7fdc7819f7a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d980 tx=0x7fdc6c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc781037f0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6c004830 con 0x7fdc781037f0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7819a3b0 con 0x7fdc781037f0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.842+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c005440 con 0x7fdc781037f0 2024-05-14T17:15:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.843+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc781a6670 con 0x7fdc781037f0 2024-05-14T17:15:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.844+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7806a230 con 0x7fdc781037f0 2024-05-14T17:15:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.845+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdc6c0055e0 con 0x7fdc781037f0 2024-05-14T17:15:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.846+0000 7fdc75ffb700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdc680b29f0 0x7fdc680b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.846+0000 7fdc77fff700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdc680b29f0 0x7fdc680b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.846+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fdc6c0e2630 con 0x7fdc781037f0 2024-05-14T17:15:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.847+0000 7fdc77fff700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdc680b29f0 0x7fdc680b4eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005950 tx=0x7fdc640058e0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:48.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:48.850+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc6c020020 con 0x7fdc781037f0 2024-05-14T17:15:49.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.107+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc7802e560 con 0x7fdc781037f0 2024-05-14T17:15:49.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.108+0000 7fdc75ffb700 1 -- 172.21.15.99:0/4063351749 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11873 (secure 0 0 0) 0x7fdc6c0a9fb0 con 0x7fdc781037f0 2024-05-14T17:15:49.108 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:15:49.108 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:15:44.595162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdc680b29f0 msgr2=0x7fdc680b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdc680b29f0 0x7fdc680b4eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005950 tx=0x7fdc640058e0 comp rx=0 tx=0).stop 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 msgr2=0x7fdc7819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 0x7fdc7819f7a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d980 tx=0x7fdc6c00dd40 comp rx=0 tx=0).stop 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 shutdown_connections 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdc680b29f0 0x7fdc680b4eb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc781088f0 0x7fdc78199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc781037f0 0x7fdc7819f7a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 --2- 172.21.15.99:0/4063351749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc781041a0 0x7fdc7819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 >> 172.21.15.99:0/4063351749 conn(0x7fdc7807a6a0 msgr2=0x7fdc781017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 shutdown_connections 2024-05-14T17:15:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:49.111+0000 7fdc7f0a5700 1 -- 172.21.15.99:0/4063351749 wait complete. 2024-05-14T17:15:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-05-14T17:15:49.402 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:49 smithi099 ceph-mon[31851]: pgmap v382: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 440 KiB/s wr, 33 op/s 2024-05-14T17:15:49.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:49 smithi150 ceph-mon[36204]: pgmap v382: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 440 KiB/s wr, 33 op/s 2024-05-14T17:15:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:49 smithi193 ceph-mon[35962]: pgmap v382: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 440 KiB/s wr, 33 op/s 2024-05-14T17:15:49.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 488453 3.95 MB/sec execute 481 sec latency 1623.680 ms 2024-05-14T17:15:49.797 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0' 2024-05-14T17:15:49.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 149.65 --await-for 45.0 2024-05-14T17:15:50.057 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:50.441 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:50 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4063351749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:50.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:50 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4063351749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:50.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:50 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4063351749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:50.558 INFO:tasks.workunit.client.0.smithi099.stdout: 1 490673 3.97 MB/sec execute 482 sec latency 1666.032 ms 2024-05-14T17:15:51.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:51 smithi099 ceph-mon[31851]: pgmap v383: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.0 MiB/s wr, 85 op/s 2024-05-14T17:15:51.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:51 smithi150 ceph-mon[36204]: pgmap v383: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.0 MiB/s wr, 85 op/s 2024-05-14T17:15:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:51 smithi193 ceph-mon[35962]: pgmap v383: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.0 MiB/s wr, 85 op/s 2024-05-14T17:15:51.571 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492000 3.97 MB/sec execute 483 sec latency 191.752 ms 2024-05-14T17:15:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 -- 172.21.15.99:0/847048269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 msgr2=0x7fa1b8101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:51.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 --2- 172.21.15.99:0/847048269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b8101fd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009990 tx=0x7fa1a8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:15:51.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 -- 172.21.15.99:0/847048269 shutdown_connections 2024-05-14T17:15:51.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 --2- 172.21.15.99:0/847048269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8102510 0x7fa1b81104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:51.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 --2- 172.21.15.99:0/847048269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8109600 0x7fa1b81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 --2- 172.21.15.99:0/847048269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b8101fd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:51.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 -- 172.21.15.99:0/847048269 >> 172.21.15.99:0/847048269 conn(0x7fa1b8078ae0 msgr2=0x7fa1b80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:51.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 -- 172.21.15.99:0/847048269 shutdown_connections 2024-05-14T17:15:51.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 -- 172.21.15.99:0/847048269 wait complete. 2024-05-14T17:15:51.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 Processor -- start 2024-05-14T17:15:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.739+0000 7fa1bffca700 1 -- start start 2024-05-14T17:15:51.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:51.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8102510 0x7fa1b819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:51.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8109600 0x7fa1b81999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:51.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81137e0 con 0x7fa1b8101b50 2024-05-14T17:15:51.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8113660 con 0x7fa1b8109600 2024-05-14T17:15:52.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8113960 con 0x7fa1b8102510 2024-05-14T17:15:52.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43812/0 (socket says 172.21.15.99:43812) 2024-05-14T17:15:52.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 -- 172.21.15.99:0/3032337707 learned_addr learned my addr 172.21.15.99:0/3032337707 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:52.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bd565700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8102510 0x7fa1b819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1be567700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8109600 0x7fa1b81999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 -- 172.21.15.99:0/3032337707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8102510 msgr2=0x7fa1b819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8102510 0x7fa1b819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 -- 172.21.15.99:0/3032337707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8109600 msgr2=0x7fa1b81999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:52.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8109600 0x7fa1b81999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 -- 172.21.15.99:0/3032337707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b819a140 con 0x7fa1b8101b50 2024-05-14T17:15:52.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bd565700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8102510 0x7fa1b819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1be567700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8109600 0x7fa1b81999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:52.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b819f820 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d980 tx=0x7fa1b400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:52.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1aeffd700 1 -- 172.21.15.99:0/3032337707 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4017070 con 0x7fa1b8101b50 2024-05-14T17:15:52.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1aeffd700 1 -- 172.21.15.99:0/3032337707 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b4004830 con 0x7fa1b8101b50 2024-05-14T17:15:52.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1aeffd700 1 -- 172.21.15.99:0/3032337707 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4005440 con 0x7fa1b8101b50 2024-05-14T17:15:52.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b819a430 con 0x7fa1b8101b50 2024-05-14T17:15:52.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.740+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a66f0 con 0x7fa1b8101b50 2024-05-14T17:15:52.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.743+0000 7fa1aeffd700 1 -- 172.21.15.99:0/3032337707 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa1b40055e0 con 0x7fa1b8101b50 2024-05-14T17:15:52.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.743+0000 7fa1aeffd700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1a40b2b20 0x7fa1a40b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.743+0000 7fa1bd565700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1a40b2b20 0x7fa1a40b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1aeffd700 1 -- 172.21.15.99:0/3032337707 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fa1b40e24e0 con 0x7fa1b8101b50 2024-05-14T17:15:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 Processor -- start 2024-05-14T17:15:52.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1bd565700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1a40b2b20 0x7fa1a40b4fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006010 tx=0x7fa1a800b540 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 -- start start 2024-05-14T17:15:52.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 0x5652c3f57cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f53150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5652c3eacf10 con 0x5652c3f578d0 2024-05-14T17:15:52.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5652c3e9cab0 con 0x5652c3f53770 2024-05-14T17:15:52.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652c3ea0af0 con 0x5652c3f581f0 2024-05-14T17:15:52.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 0x5652c3f57cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 0x5652c3f57cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43818/0 (socket says 172.21.15.99:43818) 2024-05-14T17:15:52.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.744+0000 7fa1bdd66700 1 -- 172.21.15.99:0/1945629014 learned_addr learned my addr 172.21.15.99:0/1945629014 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:52.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bd565700 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f53150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1be567700 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bdd66700 1 -- 172.21.15.99:0/1945629014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 msgr2=0x5652c3f53150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:52.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f53150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bdd66700 1 -- 172.21.15.99:0/1945629014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 msgr2=0x5652c3f53bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:52.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f53bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bdd66700 1 -- 172.21.15.99:0/1945629014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c3e46150 con 0x5652c3f578d0 2024-05-14T17:15:52.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bd565700 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f53150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:52.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 0x5652c3f57cb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40a5840 tx=0x7fa1b40a9350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d712057e6408bf2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:52.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1a3fff700 1 -- 172.21.15.99:0/1945629014 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40b2620 con 0x5652c3f578d0 2024-05-14T17:15:52.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1a3fff700 1 -- 172.21.15.99:0/1945629014 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b4004d00 con 0x5652c3f578d0 2024-05-14T17:15:52.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.745+0000 7fa1a3fff700 1 -- 172.21.15.99:0/1945629014 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40b23f0 con 0x5652c3f578d0 2024-05-14T17:15:52.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 -- 172.21.15.99:0/1945629014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 msgr2=0x5652c3f57cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 0x5652c3f57cb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40a5840 tx=0x7fa1b40a9350 comp rx=0 tx=0).stop 2024-05-14T17:15:52.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 -- 172.21.15.99:0/1945629014 shutdown_connections 2024-05-14T17:15:52.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f53bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f578d0 0x5652c3f57cb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/1945629014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f53150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 -- 172.21.15.99:0/1945629014 >> 172.21.15.99:0/1945629014 conn(0x5652c3e7bf50 msgr2=0x5652c3f49540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:52.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 -- 172.21.15.99:0/1945629014 shutdown_connections 2024-05-14T17:15:52.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 -- 172.21.15.99:0/1945629014 wait complete. 2024-05-14T17:15:52.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 -- start start 2024-05-14T17:15:52.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.746+0000 7fa1ca860b80 1 Processor -- start 2024-05-14T17:15:52.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f5f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f578d0 0x5652c3f5c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f5c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:52.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5652c3da3140 con 0x5652c3f53770 2024-05-14T17:15:52.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5652c3e04f40 con 0x5652c3f578d0 2024-05-14T17:15:52.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5652c3f5fc70 con 0x5652c3f581f0 2024-05-14T17:15:52.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 10 client.0 _ll_register_callbacks cb 0x5652c3e37af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:52.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f5f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f5f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43834/0 (socket says 172.21.15.99:43834) 2024-05-14T17:15:52.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 -- 172.21.15.99:0/2337261191 learned_addr learned my addr 172.21.15.99:0/2337261191 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:52.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bd565700 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f578d0 0x5652c3f5c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 -- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 msgr2=0x5652c3f5c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f5c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 -- 172.21.15.99:0/2337261191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f578d0 msgr2=0x5652c3f5c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:53.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f578d0 0x5652c3f5c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 -- 172.21.15.99:0/2337261191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c3deca70 con 0x5652c3f53770 2024-05-14T17:15:53.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f5f060 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40a94c0 tx=0x7fa1b400de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a460dfd116a3763 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:53.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:53.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1a17fa700 1 -- 172.21.15.99:0/2337261191 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40b2620 con 0x5652c3f53770 2024-05-14T17:15:53.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1a17fa700 1 -- 172.21.15.99:0/2337261191 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fa1b40a9c30 con 0x5652c3f53770 2024-05-14T17:15:53.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.747+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b4004d00 con 0x5652c3f53770 2024-05-14T17:15:53.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1a17fa700 1 -- 172.21.15.99:0/2337261191 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa1b40b2190 con 0x5652c3f53770 2024-05-14T17:15:53.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 10 client.16788 fetch_fsmap learned FSMap version 42 2024-05-14T17:15:53.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5652c3e56900 con 0x5652c3f53770 2024-05-14T17:15:53.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1a17fa700 1 -- 172.21.15.99:0/2337261191 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b4004d00 con 0x5652c3f53770 2024-05-14T17:15:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1a17fa700 1 -- 172.21.15.99:0/2337261191 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fa1b408fce0 con 0x5652c3f53770 2024-05-14T17:15:53.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 10 client.16788 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:15:53.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 10 client.16788 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:15:53.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 20 client.16788 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3f6c720 0x5652c3f6eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 4 client.16788 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:53.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652c3eaee60 con 0x5652c3f6c720 2024-05-14T17:15:53.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.748+0000 7fa1bd565700 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3f6c720 0x5652c3f6eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:53.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1a17fa700 10 client.16788 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1bd565700 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3f6c720 0x5652c3f6eb10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e30a2787d32f08a server_cookie=319362f0403c612c in_seq=0 out_seq=0 2024-05-14T17:15:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1a17fa700 1 -- 172.21.15.99:0/2337261191 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652c3eaee60 con 0x5652c3f6c720 2024-05-14T17:15:53.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1a17fa700 10 client.16788 handle_command_reply: tid=0 2024-05-14T17:15:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1ca860b80 1 client.16788 shutdown 2024-05-14T17:15:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 msgr2=0x5652c3f5f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:53.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.749+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f5f060 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40a94c0 tx=0x7fa1b400de60 comp rx=0 tx=0).stop 2024-05-14T17:15:53.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 shutdown_connections 2024-05-14T17:15:53.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f581f0 0x5652c3f5c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3f6c720 0x5652c3f6eb10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f5f060 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/2337261191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f578d0 0x5652c3f5c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1a17fa700 0 client.16788 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:53.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 >> 172.21.15.99:0/2337261191 conn(0x5652c3e7bf50 msgr2=0x5652c3f44060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:53.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 shutdown_connections 2024-05-14T17:15:53.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 1 -- 172.21.15.99:0/2337261191 wait complete. 2024-05-14T17:15:53.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.750+0000 7fa1ca860b80 20 client.16788 trim_cache size 0 max 16384 2024-05-14T17:15:53.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:53 smithi150 ceph-mon[36204]: pgmap v384: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 911 KiB/s wr, 71 op/s 2024-05-14T17:15:53.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 Processor -- start 2024-05-14T17:15:53.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 -- start start 2024-05-14T17:15:53.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f44170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 0x5652c3f59bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81ae3c0 con 0x5652c3f581f0 2024-05-14T17:15:53.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81abc50 con 0x5652c3f53770 2024-05-14T17:15:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.769+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81a9ae0 con 0x5652c3f578d0 2024-05-14T17:15:53.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 0x5652c3f59bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:53.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1bd565700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:53.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 0x5652c3f59bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43848/0 (socket says 172.21.15.99:43848) 2024-05-14T17:15:53.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 -- 172.21.15.99:0/4225911860 learned_addr learned my addr 172.21.15.99:0/4225911860 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:53.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f44170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:53.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 -- 172.21.15.99:0/4225911860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 msgr2=0x5652c3f596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:53.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 -- 172.21.15.99:0/4225911860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 msgr2=0x5652c3f44170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:53.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f44170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 -- 172.21.15.99:0/4225911860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c3f5a970 con 0x5652c3f581f0 2024-05-14T17:15:53.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1be567700 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 0x5652c3f59bf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009b40 tx=0x7fa1b0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e060335c161f00f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:53.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1a37fe700 1 -- 172.21.15.99:0/4225911860 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0019020 con 0x5652c3f581f0 2024-05-14T17:15:53.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1a37fe700 1 -- 172.21.15.99:0/4225911860 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b0007e00 con 0x5652c3f581f0 2024-05-14T17:15:53.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.770+0000 7fa1a37fe700 1 -- 172.21.15.99:0/4225911860 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0014720 con 0x5652c3f581f0 2024-05-14T17:15:53.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 -- 172.21.15.99:0/4225911860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 msgr2=0x5652c3f59bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:53.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 0x5652c3f59bf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009b40 tx=0x7fa1b0009e50 comp rx=0 tx=0).stop 2024-05-14T17:15:53.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 -- 172.21.15.99:0/4225911860 shutdown_connections 2024-05-14T17:15:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f581f0 0x5652c3f59bf0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f53770 0x5652c3f44170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/4225911860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:53.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 -- 172.21.15.99:0/4225911860 >> 172.21.15.99:0/4225911860 conn(0x5652c3e7bf50 msgr2=0x5652c3f56e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:53.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 -- 172.21.15.99:0/4225911860 shutdown_connections 2024-05-14T17:15:53.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 -- 172.21.15.99:0/4225911860 wait complete. 2024-05-14T17:15:53.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 -- start start 2024-05-14T17:15:53.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.771+0000 7fa1ca860b80 1 Processor -- start 2024-05-14T17:15:53.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f72060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f6cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f581f0 0x5652c3f6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:53.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81aa130 con 0x5652c3f53770 2024-05-14T17:15:53.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81ab580 con 0x5652c3f581f0 2024-05-14T17:15:53.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81aa7e0 con 0x5652c3f578d0 2024-05-14T17:15:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 10 client.0 _ll_register_callbacks cb 0x5652c3e37af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:53.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f72060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:54.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f72060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43864/0 (socket says 172.21.15.99:43864) 2024-05-14T17:15:54.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 -- 172.21.15.99:0/448893484 learned_addr learned my addr 172.21.15.99:0/448893484 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:54.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1be567700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f581f0 0x5652c3f6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:54.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bd565700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f6cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:54.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 -- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 msgr2=0x5652c3f6cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:54.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f6cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:54.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 -- 172.21.15.99:0/448893484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f581f0 msgr2=0x5652c3f6d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:54.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f581f0 0x5652c3f6d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:54.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 -- 172.21.15.99:0/448893484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c3f6d860 con 0x5652c3f53770 2024-05-14T17:15:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1bdd66700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f72060 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4021000 tx=0x7fa1b40aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f0c020f1ed45b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:54.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1a17fa700 1 -- 172.21.15.99:0/448893484 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40a5680 con 0x5652c3f53770 2024-05-14T17:15:54.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1a17fa700 1 -- 172.21.15.99:0/448893484 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fa1b40a5db0 con 0x5652c3f53770 2024-05-14T17:15:54.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.772+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b0007e00 con 0x5652c3f53770 2024-05-14T17:15:54.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1a17fa700 1 -- 172.21.15.99:0/448893484 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa1b4004d00 con 0x5652c3f53770 2024-05-14T17:15:54.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 10 client.16794 fetch_fsmap learned FSMap version 42 2024-05-14T17:15:54.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5652c3f6d860 con 0x5652c3f53770 2024-05-14T17:15:54.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1a17fa700 1 -- 172.21.15.99:0/448893484 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b0007e00 con 0x5652c3f53770 2024-05-14T17:15:54.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1a17fa700 1 -- 172.21.15.99:0/448893484 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fa1b408fce0 con 0x5652c3f53770 2024-05-14T17:15:54.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 10 client.16794 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:15:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 10 client.16794 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:15:54.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 20 client.16794 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:54.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3fdc6a0 0x5652c3fdea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:54.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 4 client.16794 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}] 2024-05-14T17:15:54.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.65, "await_for": 45.0}) -- 0x5652c3ea8640 con 0x5652c3fdc6a0 2024-05-14T17:15:54.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.773+0000 7fa1bd565700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3fdc6a0 0x5652c3fdea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:54.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.774+0000 7fa1a17fa700 10 client.16794 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:54.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.774+0000 7fa1bd565700 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3fdc6a0 0x5652c3fdea90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f22838c0963db9 server_cookie=bc814e8ea896b0e3 in_seq=0 out_seq=0 2024-05-14T17:15:54.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1a17fa700 1 -- 172.21.15.99:0/448893484 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5652c3ea8640 con 0x5652c3fdc6a0 2024-05-14T17:15:54.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1a17fa700 10 client.16794 handle_command_reply: tid=0 2024-05-14T17:15:54.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 client.16794 shutdown 2024-05-14T17:15:54.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 msgr2=0x5652c3f72060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f72060 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4021000 tx=0x7fa1b40aa000 comp rx=0 tx=0).stop 2024-05-14T17:15:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 shutdown_connections 2024-05-14T17:15:54.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652c3f581f0 0x5652c3f6d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:54.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5652c3fdc6a0 0x5652c3fdea90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:54.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652c3f53770 0x5652c3f72060 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:54.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1a17fa700 0 client.16794 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:54.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 --2- 172.21.15.99:0/448893484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5652c3f578d0 0x5652c3f6cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:54.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.834+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 >> 172.21.15.99:0/448893484 conn(0x5652c3e7bf50 msgr2=0x5652c3f5d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 shutdown_connections 2024-05-14T17:15:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1ca860b80 1 -- 172.21.15.99:0/448893484 wait complete. 2024-05-14T17:15:54.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1ca860b80 20 client.16794 trim_cache size 0 max 16384 2024-05-14T17:15:54.672 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:15:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 42, 2024-05-14T17:15:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:15:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 42, 2024-05-14T17:15:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:15:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "7f40df97": { 2024-05-14T17:15:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 42, 2024-05-14T17:15:54.730 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:15:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:15:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:15:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:15:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:15:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:15:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 149.7, 2024-05-14T17:15:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:15:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:15:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:15:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:15:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:15:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:15:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:54.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:54.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:54.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:54.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:15:54.986 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:15:55.033 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:53 smithi193 ceph-mon[35962]: pgmap v384: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 911 KiB/s wr, 71 op/s 2024-05-14T17:15:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1a40b2b20 msgr2=0x7fa1a40b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1bffca700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1a40b2b20 0x7fa1a40b4fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006010 tx=0x7fa1a800b540 comp rx=0 tx=0).stop 2024-05-14T17:15:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 msgr2=0x7fa1b819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.835+0000 7fa1bffca700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b819f820 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d980 tx=0x7fa1b400dd40 comp rx=0 tx=0).stop 2024-05-14T17:15:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 shutdown_connections 2024-05-14T17:15:55.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa1a40b2b20 0x7fa1a40b4fe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1b8109600 0x7fa1b81999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1b8101b50 0x7fa1b819f820 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 --2- 172.21.15.99:0/3032337707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b8102510 0x7fa1b819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 >> 172.21.15.99:0/3032337707 conn(0x7fa1b8078ae0 msgr2=0x7fa1b80ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:55.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 shutdown_connections 2024-05-14T17:15:55.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:15:51.836+0000 7fa1bffca700 1 -- 172.21.15.99:0/3032337707 wait complete. 2024-05-14T17:15:55.191 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.97 MB/sec execute 484 sec latency 702.449 ms 2024-05-14T17:15:55.199 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:53 smithi099 ceph-mon[31851]: pgmap v384: 97 pgs: 97 active+clean; 142 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 911 KiB/s wr, 71 op/s 2024-05-14T17:15:55.206 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.96 MB/sec execute 485 sec latency 1702.503 ms 2024-05-14T17:15:55.227 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7f40df97', quiesce took 8.3 seconds. Will release after: 21.349999999999998 2024-05-14T17:15:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.208+0000 7fb58fce1640 1 -- 172.21.15.99:0/936874016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 msgr2=0x7fb588102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.208+0000 7fb58fce1640 1 --2- 172.21.15.99:0/936874016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb588102ed0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb578009930 tx=0x7fb57802a420 comp rx=0 tx=0).stop 2024-05-14T17:15:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.208+0000 7fb58fce1640 1 -- 172.21.15.99:0/936874016 shutdown_connections 2024-05-14T17:15:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 --2- 172.21.15.99:0/936874016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb588103410 0x7fb58810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 --2- 172.21.15.99:0/936874016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb588102ed0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 --2- 172.21.15.99:0/936874016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588108a60 0x7fb588108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 -- 172.21.15.99:0/936874016 >> 172.21.15.99:0/936874016 conn(0x7fb5880fe520 msgr2=0x7fb588100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 -- 172.21.15.99:0/936874016 shutdown_connections 2024-05-14T17:15:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 -- 172.21.15.99:0/936874016 wait complete. 2024-05-14T17:15:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 Processor -- start 2024-05-14T17:15:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 -- start start 2024-05-14T17:15:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.209+0000 7fb58fce1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb5881991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588103410 0x7fb5881996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb588108a60 0x7fb58819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb588112d20 con 0x7fb588102a50 2024-05-14T17:15:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb588112ba0 con 0x7fb588108a60 2024-05-14T17:15:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb588112ea0 con 0x7fb588103410 2024-05-14T17:15:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb5881991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb5881991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43890/0 (socket says 172.21.15.99:43890) 2024-05-14T17:15:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 -- 172.21.15.99:0/819812367 learned_addr learned my addr 172.21.15.99:0/819812367 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58d27c640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588103410 0x7fb5881996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 -- 172.21.15.99:0/819812367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588103410 msgr2=0x7fb5881996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588103410 0x7fb5881996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 -- 172.21.15.99:0/819812367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb588108a60 msgr2=0x7fb58819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb588108a60 0x7fb58819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 -- 172.21.15.99:0/819812367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58819e190 con 0x7fb588102a50 2024-05-14T17:15:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58d27c640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588103410 0x7fb5881996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58da7d640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb5881991a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d770 tx=0x7fb58400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb57effd640 1 -- 172.21.15.99:0/819812367 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584017070 con 0x7fb588102a50 2024-05-14T17:15:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb57effd640 1 -- 172.21.15.99:0/819812367 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5840054d0 con 0x7fb588102a50 2024-05-14T17:15:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58819e480 con 0x7fb588102a50 2024-05-14T17:15:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb57effd640 1 -- 172.21.15.99:0/819812367 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584004c30 con 0x7fb588102a50 2024-05-14T17:15:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.210+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a5d10 con 0x7fb588102a50 2024-05-14T17:15:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.213+0000 7fb57effd640 1 -- 172.21.15.99:0/819812367 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb584004df0 con 0x7fb588102a50 2024-05-14T17:15:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.213+0000 7fb592a01740 1 Processor -- start 2024-05-14T17:15:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.213+0000 7fb592a01740 1 -- start start 2024-05-14T17:15:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb592a01740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db1dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb592a01740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb592a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb592a01740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56241da90c30 con 0x56241db1d8f0 2024-05-14T17:15:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb592a01740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56241da807d0 con 0x56241db1e210 2024-05-14T17:15:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb592a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56241da84810 con 0x56241db28ad0 2024-05-14T17:15:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db1dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db1dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43894/0 (socket says 172.21.15.99:43894) 2024-05-14T17:15:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 -- 172.21.15.99:0/2460427643 learned_addr learned my addr 172.21.15.99:0/2460427643 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb57effd640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb5740b2a90 0x7fb5740b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58d27c640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb5740b2a90 0x7fb5740b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb57effd640 1 -- 172.21.15.99:0/819812367 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb5840e20c0 con 0x7fb588102a50 2024-05-14T17:15:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58da7d640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 -- 172.21.15.99:0/2460427643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 msgr2=0x56241db2aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db2aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 -- 172.21.15.99:0/2460427643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 msgr2=0x56241db28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db28590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 -- 172.21.15.99:0/2460427643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241d795210 con 0x56241db1d8f0 2024-05-14T17:15:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58d27c640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58da7d640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.214+0000 7fb58e27e640 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db1dcd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb580015900 tx=0x7fb580015dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2424abf0b6966d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb573fff640 1 -- 172.21.15.99:0/2460427643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580022070 con 0x56241db1d8f0 2024-05-14T17:15:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb58e27e640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb5740b2a90 0x7fb5740b4f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb58001f000 tx=0x7fb58001b000 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb573fff640 1 -- 172.21.15.99:0/2460427643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb58001de30 con 0x56241db1d8f0 2024-05-14T17:15:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb573fff640 1 -- 172.21.15.99:0/2460427643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000f960 con 0x56241db1d8f0 2024-05-14T17:15:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb592a01740 1 -- 172.21.15.99:0/2460427643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 msgr2=0x56241db1dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb592a01740 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db1dcd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb580015900 tx=0x7fb580015dc0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb592a01740 1 -- 172.21.15.99:0/2460427643 shutdown_connections 2024-05-14T17:15:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb592a01740 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb592a01740 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.215+0000 7fb592a01740 1 --2- 172.21.15.99:0/2460427643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db1dcd0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- 172.21.15.99:0/2460427643 >> 172.21.15.99:0/2460427643 conn(0x56241db19e00 msgr2=0x56241db1a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- 172.21.15.99:0/2460427643 shutdown_connections 2024-05-14T17:15:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- 172.21.15.99:0/2460427643 wait complete. 2024-05-14T17:15:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- start start 2024-05-14T17:15:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 Processor -- start 2024-05-14T17:15:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1d8f0 0x56241db28490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db25780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 0x56241db25d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56241d766790 con 0x56241db28ad0 2024-05-14T17:15:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56241da3c890 con 0x56241db1e210 2024-05-14T17:15:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.216+0000 7fb592a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56241db2b7a0 con 0x56241db1d8f0 2024-05-14T17:15:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb592a01740 10 client.0 _ll_register_callbacks cb 0x56241da3fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 0x56241db25d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 0x56241db25d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43910/0 (socket says 172.21.15.99:43910) 2024-05-14T17:15:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 -- 172.21.15.99:0/2991335764 learned_addr learned my addr 172.21.15.99:0/2991335764 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58da7d640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1d8f0 0x56241db28490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58d27c640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db25780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 -- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1d8f0 msgr2=0x56241db28490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1d8f0 0x56241db28490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 -- 172.21.15.99:0/2991335764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 msgr2=0x56241db25780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db25780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 -- 172.21.15.99:0/2991335764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241d861620 con 0x56241db28ad0 2024-05-14T17:15:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58d27c640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db25780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb58e27e640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 0x56241db25d90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb580009e60 tx=0x7fb5800097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefee70c05ecae29 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb5717fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb5717fa640 1 -- 172.21.15.99:0/2991335764 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58002b070 con 0x56241db28ad0 2024-05-14T17:15:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb5717fa640 1 -- 172.21.15.99:0/2991335764 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb58001db90 con 0x56241db28ad0 2024-05-14T17:15:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58001de30 con 0x56241db28ad0 2024-05-14T17:15:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb5717fa640 1 -- 172.21.15.99:0/2991335764 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb580022040 con 0x56241db28ad0 2024-05-14T17:15:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb592a01740 10 client.16806 fetch_fsmap learned FSMap version 42 2024-05-14T17:15:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.217+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56241db37170 con 0x56241db28ad0 2024-05-14T17:15:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb5717fa640 1 -- 172.21.15.99:0/2991335764 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb58001de30 con 0x56241db28ad0 2024-05-14T17:15:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb5717fa640 1 -- 172.21.15.99:0/2991335764 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fb5800276b0 con 0x56241db28ad0 2024-05-14T17:15:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb592a01740 10 client.16806 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:15:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb592a01740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:56.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb592a01740 20 client.16806 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb592a01740 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241db38630 0x56241db3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb592a01740 4 client.16806 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56241da92b80 con 0x56241db38630 2024-05-14T17:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.218+0000 7fb58da7d640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241db38630 0x56241db3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.219+0000 7fb5717fa640 10 client.16806 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.219+0000 7fb58da7d640 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241db38630 0x56241db3aa20 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b8e8f99aadbdda server_cookie=e365e51e5fb6500f in_seq=0 out_seq=0 2024-05-14T17:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.219+0000 7fb5717fa640 1 -- 172.21.15.99:0/2991335764 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56241da92b80 con 0x56241db38630 2024-05-14T17:15:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.219+0000 7fb5717fa640 10 client.16806 handle_command_reply: tid=0 2024-05-14T17:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 client.16806 shutdown 2024-05-14T17:15:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 msgr2=0x56241db25d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 0x56241db25d90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb580009e60 tx=0x7fb5800097f0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 shutdown_connections 2024-05-14T17:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241db38630 0x56241db3aa20 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db28ad0 0x56241db25d90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1e210 0x56241db25780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb5717fa640 0 client.16806 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 --2- 172.21.15.99:0/2991335764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1d8f0 0x56241db28490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 >> 172.21.15.99:0/2991335764 conn(0x56241db19e00 msgr2=0x56241db19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 shutdown_connections 2024-05-14T17:15:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 1 -- 172.21.15.99:0/2991335764 wait complete. 2024-05-14T17:15:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.220+0000 7fb592a01740 20 client.16806 trim_cache size 0 max 16384 2024-05-14T17:15:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 Processor -- start 2024-05-14T17:15:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 -- start start 2024-05-14T17:15:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1d8f0 0x56241db34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 0x56241db27fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b0f40 con 0x56241db1e210 2024-05-14T17:15:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ae810 con 0x56241db1d8f0 2024-05-14T17:15:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb592a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ac680 con 0x56241db28ad0 2024-05-14T17:15:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb58d27c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 0x56241db27fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb58d27c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 0x56241db27fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43918/0 (socket says 172.21.15.99:43918) 2024-05-14T17:15:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.235+0000 7fb58d27c640 1 -- 172.21.15.99:0/3042267167 learned_addr learned my addr 172.21.15.99:0/3042267167 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58da7d640 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1d8f0 0x56241db34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58e27e640 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:57.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58d27c640 1 -- 172.21.15.99:0/3042267167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 msgr2=0x56241db401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58d27c640 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58d27c640 1 -- 172.21.15.99:0/3042267167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1d8f0 msgr2=0x56241db34670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58d27c640 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1d8f0 0x56241db34670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58d27c640 1 -- 172.21.15.99:0/3042267167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241db3af80 con 0x56241db1e210 2024-05-14T17:15:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58da7d640 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1d8f0 0x56241db34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb58d27c640 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 0x56241db27fc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb57802abd0 tx=0x7fb578032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bda5714a7dade5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb5737fe640 1 -- 172.21.15.99:0/3042267167 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57803f070 con 0x56241db1e210 2024-05-14T17:15:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb5737fe640 1 -- 172.21.15.99:0/3042267167 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578032e50 con 0x56241db1e210 2024-05-14T17:15:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.236+0000 7fb5737fe640 1 -- 172.21.15.99:0/3042267167 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578035070 con 0x56241db1e210 2024-05-14T17:15:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- 172.21.15.99:0/3042267167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 msgr2=0x56241db27fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 0x56241db27fc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb57802abd0 tx=0x7fb578032c60 comp rx=0 tx=0).stop 2024-05-14T17:15:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- 172.21.15.99:0/3042267167 shutdown_connections 2024-05-14T17:15:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db28ad0 0x56241db401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1e210 0x56241db27fc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- 172.21.15.99:0/3042267167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db1d8f0 0x56241db34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- 172.21.15.99:0/3042267167 >> 172.21.15.99:0/3042267167 conn(0x56241db19e00 msgr2=0x56241db1fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- 172.21.15.99:0/3042267167 shutdown_connections 2024-05-14T17:15:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- 172.21.15.99:0/3042267167 wait complete. 2024-05-14T17:15:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- start start 2024-05-14T17:15:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 Processor -- start 2024-05-14T17:15:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db30ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1e210 0x56241db2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db28ad0 0x56241db2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881accd0 con 0x56241db1d8f0 2024-05-14T17:15:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ae140 con 0x56241db28ad0 2024-05-14T17:15:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.237+0000 7fb592a01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881ad3a0 con 0x56241db1e210 2024-05-14T17:15:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb592a01740 10 client.0 _ll_register_callbacks cb 0x7fb580015580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db30ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db30ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43932/0 (socket says 172.21.15.99:43932) 2024-05-14T17:15:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 -- 172.21.15.99:0/650614116 learned_addr learned my addr 172.21.15.99:0/650614116 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:15:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58d27c640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1e210 0x56241db2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58e27e640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db28ad0 0x56241db2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 -- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1e210 msgr2=0x56241db2c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1e210 0x56241db2c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 -- 172.21.15.99:0/650614116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db28ad0 msgr2=0x56241db2c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db28ad0 0x56241db2c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 -- 172.21.15.99:0/650614116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241db21360 con 0x56241db1d8f0 2024-05-14T17:15:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb58da7d640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db30ec0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb5840a5420 tx=0x7fb58400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af47ef737a5612df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb5717fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:15:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb5717fa640 1 -- 172.21.15.99:0/650614116 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584020040 con 0x56241db1d8f0 2024-05-14T17:15:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb5717fa640 1 -- 172.21.15.99:0/650614116 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb5840a5630 con 0x56241db1d8f0 2024-05-14T17:15:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578032e50 con 0x56241db1d8f0 2024-05-14T17:15:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb5717fa640 1 -- 172.21.15.99:0/650614116 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb5840a93a0 con 0x56241db1d8f0 2024-05-14T17:15:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb592a01740 10 client.16812 fetch_fsmap learned FSMap version 42 2024-05-14T17:15:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56241dba2b20 con 0x56241db1d8f0 2024-05-14T17:15:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.238+0000 7fb5717fa640 1 -- 172.21.15.99:0/650614116 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578032e50 con 0x56241db1d8f0 2024-05-14T17:15:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb5717fa640 1 -- 172.21.15.99:0/650614116 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fb5840e40b0 con 0x56241db1d8f0 2024-05-14T17:15:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb592a01740 10 client.16812 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:15:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb592a01740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:15:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb592a01740 20 client.16812 populate_metadata read hostname 'smithi099' 2024-05-14T17:15:57.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb592a01740 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241dba3240 0x56241dba5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb592a01740 4 client.16812 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56241da8c360 con 0x56241dba3240 2024-05-14T17:15:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb58d27c640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241dba3240 0x56241dba5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb5717fa640 10 client.16812 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.239+0000 7fb58d27c640 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241dba3240 0x56241dba5660 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95637cca97a016d0 server_cookie=3f864fb1979be3ea in_seq=0 out_seq=0 2024-05-14T17:15:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.277+0000 7fb5717fa640 1 -- 172.21.15.99:0/650614116 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56241da8c360 con 0x56241dba3240 2024-05-14T17:15:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.277+0000 7fb5717fa640 10 client.16812 handle_command_reply: tid=0 2024-05-14T17:15:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.277+0000 7fb592a01740 1 client.16812 shutdown 2024-05-14T17:15:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 msgr2=0x56241db30ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db30ec0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb5840a5420 tx=0x7fb58400b480 comp rx=0 tx=0).stop 2024-05-14T17:15:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 shutdown_connections 2024-05-14T17:15:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x56241dba3240 0x56241dba5660 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56241db28ad0 0x56241db2c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb5717fa640 0 client.16812 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:15:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56241db1e210 0x56241db2c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 --2- 172.21.15.99:0/650614116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56241db1d8f0 0x56241db30ec0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 >> 172.21.15.99:0/650614116 conn(0x56241db19e00 msgr2=0x56241db1f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 shutdown_connections 2024-05-14T17:15:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 1 -- 172.21.15.99:0/650614116 wait complete. 2024-05-14T17:15:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb592a01740 20 client.16812 trim_cache size 0 max 16384 2024-05-14T17:15:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:15:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:15:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:15:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb5740b2a90 msgr2=0x7fb5740b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:58.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb5740b2a90 0x7fb5740b4f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb58001f000 tx=0x7fb58001b000 comp rx=0 tx=0).stop 2024-05-14T17:15:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 msgr2=0x7fb5881991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb5881991a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d770 tx=0x7fb58400dc30 comp rx=0 tx=0).stop 2024-05-14T17:15:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 shutdown_connections 2024-05-14T17:15:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fb5740b2a90 0x7fb5740b4f50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb588108a60 0x7fb58819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb588103410 0x7fb5881996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 --2- 172.21.15.99:0/819812367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588102a50 0x7fb5881991a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.278+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 >> 172.21.15.99:0/819812367 conn(0x7fb5880fe520 msgr2=0x7fb5880feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.279+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 shutdown_connections 2024-05-14T17:15:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:15:54.279+0000 7fb58fce1640 1 -- 172.21.15.99:0/819812367 wait complete. 2024-05-14T17:15:58.327 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.95 MB/sec execute 486 sec latency 2702.546 ms 2024-05-14T17:15:58.334 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:55 smithi099 ceph-mon[31851]: pgmap v385: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-05-14T17:15:58.342 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:55 smithi193 ceph-mon[35962]: pgmap v385: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-05-14T17:15:58.351 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:57 smithi193 ceph-mon[35962]: pgmap v386: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-05-14T17:15:58.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:55 smithi150 ceph-mon[36204]: pgmap v385: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-05-14T17:15:58.364 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:57 smithi150 ceph-mon[36204]: pgmap v386: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-05-14T17:15:58.374 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.94 MB/sec execute 487 sec latency 3702.615 ms 2024-05-14T17:15:58.402 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.93 MB/sec execute 488 sec latency 4702.710 ms 2024-05-14T17:15:58.410 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:57 smithi099 ceph-mon[31851]: pgmap v386: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-05-14T17:15:58.418 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.92 MB/sec execute 489 sec latency 5702.832 ms 2024-05-14T17:15:58.559 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.92 MB/sec execute 490 sec latency 6702.931 ms 2024-05-14T17:15:59.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:15:59 smithi150 ceph-mon[36204]: pgmap v387: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-05-14T17:15:59.524 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:15:59 smithi099 ceph-mon[31851]: pgmap v387: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-05-14T17:15:59.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:15:59 smithi193 ceph-mon[35962]: pgmap v387: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-05-14T17:15:59.583 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.91 MB/sec execute 491 sec latency 7702.978 ms 2024-05-14T17:16:00.559 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.90 MB/sec execute 492 sec latency 8703.033 ms 2024-05-14T17:16:01.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:01 smithi150 ceph-mon[36204]: pgmap v388: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-05-14T17:16:01.460 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:01 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:01.468 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:01 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:01.476 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:01 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:01.484 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:01 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:16:01.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:01 smithi099 ceph-mon[31851]: pgmap v388: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-05-14T17:16:01.520 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:01 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:01.527 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:01 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:01.534 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:01 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:01.542 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:01 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:16:01.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:01 smithi193 ceph-mon[35962]: pgmap v388: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-05-14T17:16:01.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:01 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:01.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:01 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:01.622 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:01 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:01.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:01 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:16:01.674 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.89 MB/sec execute 493 sec latency 9703.134 ms 2024-05-14T17:16:02.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.88 MB/sec execute 494 sec latency 10703.228 ms 2024-05-14T17:16:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:03 smithi150 ceph-mon[36204]: pgmap v389: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 545 KiB/s wr, 67 op/s 2024-05-14T17:16:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:03 smithi099 ceph-mon[31851]: pgmap v389: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 545 KiB/s wr, 67 op/s 2024-05-14T17:16:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:03 smithi193 ceph-mon[35962]: pgmap v389: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 545 KiB/s wr, 67 op/s 2024-05-14T17:16:03.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.88 MB/sec execute 495 sec latency 11703.329 ms 2024-05-14T17:16:04.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.87 MB/sec execute 496 sec latency 12703.426 ms 2024-05-14T17:16:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:05 smithi150 ceph-mon[36204]: pgmap v390: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 597 KiB/s wr, 88 op/s 2024-05-14T17:16:05.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:05 smithi099 ceph-mon[31851]: pgmap v390: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 597 KiB/s wr, 88 op/s 2024-05-14T17:16:05.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:05 smithi193 ceph-mon[35962]: pgmap v390: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 597 KiB/s wr, 88 op/s 2024-05-14T17:16:05.587 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.86 MB/sec execute 497 sec latency 13703.523 ms 2024-05-14T17:16:06.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.85 MB/sec execute 498 sec latency 14703.597 ms 2024-05-14T17:16:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.375+0000 7f7a282c8640 1 -- 172.21.15.99:0/2505478449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20107a70 msgr2=0x7f7a20107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.375+0000 7f7a282c8640 1 --2- 172.21.15.99:0/2505478449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20107a70 0x7f7a20107e50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009cd0 tx=0x7f7a1c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:16:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- 172.21.15.99:0/2505478449 shutdown_connections 2024-05-14T17:16:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 --2- 172.21.15.99:0/2505478449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a201025f0 0x7f7a2010e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 --2- 172.21.15.99:0/2505478449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a20101c30 0x7f7a201020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 --2- 172.21.15.99:0/2505478449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20107a70 0x7f7a20107e50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- 172.21.15.99:0/2505478449 >> 172.21.15.99:0/2505478449 conn(0x7f7a200786a0 msgr2=0x7f7a20078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- 172.21.15.99:0/2505478449 shutdown_connections 2024-05-14T17:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- 172.21.15.99:0/2505478449 wait complete. 2024-05-14T17:16:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 Processor -- start 2024-05-14T17:16:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- start start 2024-05-14T17:16:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a20101c30 0x7f7a20199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 0x7f7a2019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a20107a70 0x7f7a201a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111bf0 con 0x7f7a201025f0 2024-05-14T17:16:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111a70 con 0x7f7a20107a70 2024-05-14T17:16:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.376+0000 7f7a282c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111d70 con 0x7f7a20101c30 2024-05-14T17:16:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 0x7f7a2019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 0x7f7a2019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57302/0 (socket says 172.21.15.99:57302) 2024-05-14T17:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 -- 172.21.15.99:0/854051202 learned_addr learned my addr 172.21.15.99:0/854051202 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a26064640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a20101c30 0x7f7a20199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 -- 172.21.15.99:0/854051202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a20101c30 msgr2=0x7f7a20199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a26865640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a20107a70 0x7f7a201a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a20101c30 0x7f7a20199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 -- 172.21.15.99:0/854051202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a20107a70 msgr2=0x7f7a201a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a20107a70 0x7f7a201a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 -- 172.21.15.99:0/854051202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a201a55b0 con 0x7f7a201025f0 2024-05-14T17:16:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a25863640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 0x7f7a2019a530 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d770 tx=0x7f7a1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a177fe640 1 -- 172.21.15.99:0/854051202 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10017070 con 0x7f7a201025f0 2024-05-14T17:16:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a177fe640 1 -- 172.21.15.99:0/854051202 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a100054d0 con 0x7f7a201025f0 2024-05-14T17:16:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a201a5780 con 0x7f7a201025f0 2024-05-14T17:16:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a177fe640 1 -- 172.21.15.99:0/854051202 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10004c30 con 0x7f7a201025f0 2024-05-14T17:16:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.377+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a201a5c70 con 0x7f7a201025f0 2024-05-14T17:16:07.796 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:07 smithi193 ceph-mon[35962]: pgmap v391: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 80 KiB/s wr, 40 op/s 2024-05-14T17:16:07.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:07 smithi150 ceph-mon[36204]: pgmap v391: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 80 KiB/s wr, 40 op/s 2024-05-14T17:16:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 Processor -- start 2024-05-14T17:16:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 -- start start 2024-05-14T17:16:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a177fe640 1 -- 172.21.15.99:0/854051202 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7a10004e10 con 0x7f7a201025f0 2024-05-14T17:16:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 0x5572468e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55724684ec30 con 0x5572468e6ad0 2024-05-14T17:16:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55724683e7d0 con 0x5572468db8f0 2024-05-14T17:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557246842810 con 0x5572468dc210 2024-05-14T17:16:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 0x5572468e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 0x5572468e8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57316/0 (socket says 172.21.15.99:57316) 2024-05-14T17:16:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a25863640 1 -- 172.21.15.99:0/328820243 learned_addr learned my addr 172.21.15.99:0/328820243 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a26064640 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.381+0000 7f7a26865640 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a25863640 1 -- 172.21.15.99:0/328820243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 msgr2=0x5572468e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a25863640 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a25863640 1 -- 172.21.15.99:0/328820243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 msgr2=0x5572468dbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a25863640 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a25863640 1 -- 172.21.15.99:0/328820243 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557246553210 con 0x5572468e6ad0 2024-05-14T17:16:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a26865640 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a25863640 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 0x5572468e8ec0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7a100df6b0 tx=0x7f7a100ea6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b228556423b2c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a177fe640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7a0c0b2a90 0x7f7a0c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a14bfe640 1 -- 172.21.15.99:0/328820243 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100f5070 con 0x5572468e6ad0 2024-05-14T17:16:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a14bfe640 1 -- 172.21.15.99:0/328820243 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a100a8de0 con 0x5572468e6ad0 2024-05-14T17:16:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a26865640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7a0c0b2a90 0x7f7a0c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a14bfe640 1 -- 172.21.15.99:0/328820243 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100a8a40 con 0x5572468e6ad0 2024-05-14T17:16:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.382+0000 7f7a177fe640 1 -- 172.21.15.99:0/854051202 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f7a100e1ff0 con 0x7f7a201025f0 2024-05-14T17:16:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 -- 172.21.15.99:0/328820243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 msgr2=0x5572468e8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a26865640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7a0c0b2a90 0x7f7a0c0b4f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005940 tx=0x7f7a18005e50 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 0x5572468e8ec0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7a100df6b0 tx=0x7f7a100ea6f0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 -- 172.21.15.99:0/328820243 shutdown_connections 2024-05-14T17:16:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468e6ad0 0x5572468e8ec0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/328820243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 -- 172.21.15.99:0/328820243 >> 172.21.15.99:0/328820243 conn(0x5572468d7e00 msgr2=0x5572468d81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 -- 172.21.15.99:0/328820243 shutdown_connections 2024-05-14T17:16:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 -- 172.21.15.99:0/328820243 wait complete. 2024-05-14T17:16:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.383+0000 7f7a2afe7740 1 -- start start 2024-05-14T17:16:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 Processor -- start 2024-05-14T17:16:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 0x5572468e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 0x5572468e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557246524790 con 0x5572468db8f0 2024-05-14T17:16:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5572467fa890 con 0x5572468e6ad0 2024-05-14T17:16:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572468e97a0 con 0x5572468dc210 2024-05-14T17:16:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a2afe7740 10 client.0 _ll_register_callbacks cb 0x5572467fdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 0x5572468e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 0x5572468e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57328/0 (socket says 172.21.15.99:57328) 2024-05-14T17:16:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 -- 172.21.15.99:0/1789701991 learned_addr learned my addr 172.21.15.99:0/1789701991 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a25863640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 -- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 msgr2=0x5572468e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 -- 172.21.15.99:0/1789701991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 msgr2=0x5572468e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 0x5572468e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 -- 172.21.15.99:0/1789701991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724661f620 con 0x5572468db8f0 2024-05-14T17:16:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a25863640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a26064640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 0x5572468e63d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c02ae00 tx=0x7f7a1c031c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404dfdc5ff68f5cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a09ffb640 1 -- 172.21.15.99:0/1789701991 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c04a060 con 0x5572468db8f0 2024-05-14T17:16:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.384+0000 7f7a09ffb640 1 -- 172.21.15.99:0/1789701991 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f7a1c003b80 con 0x5572468db8f0 2024-05-14T17:16:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a100a8de0 con 0x5572468db8f0 2024-05-14T17:16:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a09ffb640 1 -- 172.21.15.99:0/1789701991 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c003d80 con 0x5572468db8f0 2024-05-14T17:16:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 10 client.16824 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5572468f5160 con 0x5572468db8f0 2024-05-14T17:16:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a09ffb640 1 -- 172.21.15.99:0/1789701991 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a100a8de0 con 0x5572468db8f0 2024-05-14T17:16:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a09ffb640 1 -- 172.21.15.99:0/1789701991 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f7a1c030070 con 0x5572468db8f0 2024-05-14T17:16:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 10 client.16824 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 20 client.16824 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5572468f6670 0x5572468f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 4 client.16824 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:16:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557246850b80 con 0x5572468f6670 2024-05-14T17:16:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.385+0000 7f7a25863640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5572468f6670 0x5572468f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.386+0000 7f7a09ffb640 10 client.16824 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.386+0000 7f7a25863640 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5572468f6670 0x5572468f8a60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97dd9850cb01e1bc server_cookie=5f5ab64cfb8381aa in_seq=0 out_seq=0 2024-05-14T17:16:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a09ffb640 1 -- 172.21.15.99:0/1789701991 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557246850b80 con 0x5572468f6670 2024-05-14T17:16:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a09ffb640 10 client.16824 handle_command_reply: tid=0 2024-05-14T17:16:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 client.16824 shutdown 2024-05-14T17:16:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 msgr2=0x5572468e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 0x5572468e63d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c02ae00 tx=0x7f7a1c031c10 comp rx=0 tx=0).stop 2024-05-14T17:16:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 shutdown_connections 2024-05-14T17:16:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5572468f6670 0x5572468f8a60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 0x5572468e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468dc210 0x5572468e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a09ffb640 0 client.16824 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/1789701991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468db8f0 0x5572468e63d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 >> 172.21.15.99:0/1789701991 conn(0x5572468d7e00 msgr2=0x5572468d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:09.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 shutdown_connections 2024-05-14T17:16:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 1 -- 172.21.15.99:0/1789701991 wait complete. 2024-05-14T17:16:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.387+0000 7f7a2afe7740 20 client.16824 trim_cache size 0 max 16384 2024-05-14T17:16:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 Processor -- start 2024-05-14T17:16:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 -- start start 2024-05-14T17:16:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468e6ad0 0x5572468d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201b0f10 con 0x5572468dc210 2024-05-14T17:16:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ae7e0 con 0x5572468db8f0 2024-05-14T17:16:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ac650 con 0x5572468e6ad0 2024-05-14T17:16:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468e5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57332/0 (socket says 172.21.15.99:57332) 2024-05-14T17:16:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 -- 172.21.15.99:0/2166844343 learned_addr learned my addr 172.21.15.99:0/2166844343 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a26064640 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 -- 172.21.15.99:0/2166844343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468e6ad0 msgr2=0x5572468d6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468e6ad0 0x5572468d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 -- 172.21.15.99:0/2166844343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 msgr2=0x5572468dedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.402+0000 7f7a25863640 1 -- 172.21.15.99:0/2166844343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572468f8dd0 con 0x5572468dc210 2024-05-14T17:16:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a25863640 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468e5f00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7a100a5350 tx=0x7f7a10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca129245ea3b7a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a0bfff640 1 -- 172.21.15.99:0/2166844343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10076930 con 0x5572468dc210 2024-05-14T17:16:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a0bfff640 1 -- 172.21.15.99:0/2166844343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a10005680 con 0x5572468dc210 2024-05-14T17:16:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a0bfff640 1 -- 172.21.15.99:0/2166844343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10076b10 con 0x5572468dc210 2024-05-14T17:16:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2166844343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 msgr2=0x5572468e5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468e5f00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7a100a5350 tx=0x7f7a10009f90 comp rx=0 tx=0).stop 2024-05-14T17:16:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2166844343 shutdown_connections 2024-05-14T17:16:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468e6ad0 0x5572468d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468e5f00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2166844343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468db8f0 0x5572468dedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2166844343 >> 172.21.15.99:0/2166844343 conn(0x5572468d7e00 msgr2=0x5572468d6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2166844343 shutdown_connections 2024-05-14T17:16:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.403+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2166844343 wait complete. 2024-05-14T17:16:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 -- start start 2024-05-14T17:16:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 Processor -- start 2024-05-14T17:16:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468db8f0 0x5572468eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468ef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 0x5572468ea080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201acca0 con 0x5572468dc210 2024-05-14T17:16:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ae110 con 0x5572468e6ad0 2024-05-14T17:16:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ad370 con 0x5572468db8f0 2024-05-14T17:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a2afe7740 10 client.0 _ll_register_callbacks cb 0x7f7a1c0318a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468ef040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57340/0 (socket says 172.21.15.99:57340) 2024-05-14T17:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 -- 172.21.15.99:0/2846904930 learned_addr learned my addr 172.21.15.99:0/2846904930 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a26064640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468db8f0 0x5572468eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 -- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468db8f0 msgr2=0x5572468eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468db8f0 0x5572468eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 -- 172.21.15.99:0/2846904930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 msgr2=0x5572468ea080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 0x5572468ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 -- 172.21.15.99:0/2846904930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572468ea880 con 0x5572468dc210 2024-05-14T17:16:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a26064640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468db8f0 0x5572468eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a25863640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468ef040 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7a100a8fd0 tx=0x7f7a100eacf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cb5f22ea7adcac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.404+0000 7f7a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a09ffb640 1 -- 172.21.15.99:0/2846904930 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10076d00 con 0x5572468dc210 2024-05-14T17:16:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a09ffb640 1 -- 172.21.15.99:0/2846904930 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f7a100e5650 con 0x5572468dc210 2024-05-14T17:16:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a10005680 con 0x5572468dc210 2024-05-14T17:16:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a09ffb640 1 -- 172.21.15.99:0/2846904930 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7a10020020 con 0x5572468dc210 2024-05-14T17:16:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 10 client.16830 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557246960ad0 con 0x5572468dc210 2024-05-14T17:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a09ffb640 1 -- 172.21.15.99:0/2846904930 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a10005680 con 0x5572468dc210 2024-05-14T17:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a09ffb640 1 -- 172.21.15.99:0/2846904930 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f7a1007dc70 con 0x5572468dc210 2024-05-14T17:16:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 10 client.16830 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 20 client.16830 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x557246960e00 0x5572469631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 4 client.16830 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:16:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55724684a360 con 0x557246960e00 2024-05-14T17:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.405+0000 7f7a26064640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x557246960e00 0x5572469631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.406+0000 7f7a09ffb640 10 client.16830 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.406+0000 7f7a26064640 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x557246960e00 0x5572469631f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832d433094798f9e server_cookie=984f47621874301a in_seq=0 out_seq=0 2024-05-14T17:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a09ffb640 1 -- 172.21.15.99:0/2846904930 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55724684a360 con 0x557246960e00 2024-05-14T17:16:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a09ffb640 10 client.16830 handle_command_reply: tid=0 2024-05-14T17:16:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 client.16830 shutdown 2024-05-14T17:16:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 msgr2=0x5572468ef040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468ef040 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7a100a8fd0 tx=0x7f7a100eacf0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 shutdown_connections 2024-05-14T17:16:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x557246960e00 0x5572469631f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5572468e6ad0 0x5572468ea080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572468dc210 0x5572468ef040 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a09ffb640 0 client.16830 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 --2- 172.21.15.99:0/2846904930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572468db8f0 0x5572468eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 >> 172.21.15.99:0/2846904930 conn(0x5572468d7e00 msgr2=0x5572468f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 shutdown_connections 2024-05-14T17:16:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 1 -- 172.21.15.99:0/2846904930 wait complete. 2024-05-14T17:16:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a2afe7740 20 client.16830 trim_cache size 0 max 16384 2024-05-14T17:16:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:16:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:16:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:16:10.514 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:09 smithi193 ceph-mon[35962]: pgmap v392: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 80 KiB/s wr, 40 op/s 2024-05-14T17:16:10.522 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:09 smithi150 ceph-mon[36204]: pgmap v392: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 80 KiB/s wr, 40 op/s 2024-05-14T17:16:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7a0c0b2a90 msgr2=0x7f7a0c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7a0c0b2a90 0x7f7a0c0b4f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005940 tx=0x7f7a18005e50 comp rx=0 tx=0).stop 2024-05-14T17:16:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 msgr2=0x7f7a2019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 0x7f7a2019a530 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d770 tx=0x7f7a1000dc30 comp rx=0 tx=0).stop 2024-05-14T17:16:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 shutdown_connections 2024-05-14T17:16:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7a0c0b2a90 0x7f7a0c0b4f50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a20107a70 0x7f7a201a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a201025f0 0x7f7a2019a530 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 --2- 172.21.15.99:0/854051202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a20101c30 0x7f7a20199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.440+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 >> 172.21.15.99:0/854051202 conn(0x7f7a200786a0 msgr2=0x7f7a20100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.441+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 shutdown_connections 2024-05-14T17:16:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:07.441+0000 7f7a282c8640 1 -- 172.21.15.99:0/854051202 wait complete. 2024-05-14T17:16:10.723 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:07 smithi099 ceph-mon[31851]: pgmap v391: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 80 KiB/s wr, 40 op/s 2024-05-14T17:16:10.735 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.85 MB/sec execute 499 sec latency 15703.662 ms 2024-05-14T17:16:10.748 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.84 MB/sec execute 500 sec latency 16703.764 ms 2024-05-14T17:16:10.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:09 smithi099 ceph-mon[31851]: pgmap v392: 97 pgs: 97 active+clean; 141 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 80 KiB/s wr, 40 op/s 2024-05-14T17:16:10.774 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.83 MB/sec execute 501 sec latency 17703.825 ms 2024-05-14T17:16:10.782 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.82 MB/sec execute 502 sec latency 18703.865 ms 2024-05-14T17:16:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:11 smithi150 ceph-mon[36204]: pgmap v393: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 657 KiB/s wr, 61 op/s 2024-05-14T17:16:11.470 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:11 smithi150 ceph-mon[36204]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:16:11.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:11 smithi099 ceph-mon[31851]: pgmap v393: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 657 KiB/s wr, 61 op/s 2024-05-14T17:16:11.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:11 smithi099 ceph-mon[31851]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:16:11.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:11 smithi193 ceph-mon[35962]: pgmap v393: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 657 KiB/s wr, 61 op/s 2024-05-14T17:16:11.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:11 smithi193 ceph-mon[35962]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:16:11.582 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.82 MB/sec execute 503 sec latency 19703.954 ms 2024-05-14T17:16:12.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.81 MB/sec execute 504 sec latency 20704.028 ms 2024-05-14T17:16:13.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:13 smithi150 ceph-mon[36204]: pgmap v395: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 756 KiB/s wr, 50 op/s 2024-05-14T17:16:13.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:13 smithi099 ceph-mon[31851]: pgmap v395: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 756 KiB/s wr, 50 op/s 2024-05-14T17:16:13.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:13 smithi193 ceph-mon[35962]: pgmap v395: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 756 KiB/s wr, 50 op/s 2024-05-14T17:16:13.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.80 MB/sec execute 505 sec latency 21704.118 ms 2024-05-14T17:16:14.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.79 MB/sec execute 506 sec latency 22704.157 ms 2024-05-14T17:16:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:15 smithi150 ceph-mon[36204]: pgmap v396: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 733 KiB/s wr, 36 op/s 2024-05-14T17:16:15.472 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:15 smithi150 ceph-mon[36204]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:16:15.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:15 smithi099 ceph-mon[31851]: pgmap v396: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 733 KiB/s wr, 36 op/s 2024-05-14T17:16:15.520 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:15 smithi099 ceph-mon[31851]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:16:15.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:15 smithi193 ceph-mon[35962]: pgmap v396: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 733 KiB/s wr, 36 op/s 2024-05-14T17:16:15.549 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:15 smithi193 ceph-mon[35962]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:16:15.560 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.79 MB/sec execute 507 sec latency 23704.270 ms 2024-05-14T17:16:16.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.78 MB/sec execute 508 sec latency 24704.415 ms 2024-05-14T17:16:16.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:16:16.864 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:17.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:17 smithi099 ceph-mon[31851]: pgmap v398: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 916 KiB/s wr, 45 op/s 2024-05-14T17:16:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:17 smithi150 ceph-mon[36204]: pgmap v398: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 916 KiB/s wr, 45 op/s 2024-05-14T17:16:17.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:17 smithi193 ceph-mon[35962]: pgmap v398: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 916 KiB/s wr, 45 op/s 2024-05-14T17:16:17.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.77 MB/sec execute 509 sec latency 25704.504 ms 2024-05-14T17:16:18.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.76 MB/sec execute 510 sec latency 26704.546 ms 2024-05-14T17:16:18.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 -- 172.21.15.99:0/108355185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 msgr2=0x7f78b010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:18.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 --2- 172.21.15.99:0/108355185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b010f2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b0a0 tx=0x7f78a800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:16:18.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 -- 172.21.15.99:0/108355185 shutdown_connections 2024-05-14T17:16:18.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 --2- 172.21.15.99:0/108355185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b010f2f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:18.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 --2- 172.21.15.99:0/108355185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b0108520 0x7f78b0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:18.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 --2- 172.21.15.99:0/108355185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78b0102440 0x7f78b01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:18.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 -- 172.21.15.99:0/108355185 >> 172.21.15.99:0/108355185 conn(0x7f78b00793f0 msgr2=0x7f78b0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:18.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 -- 172.21.15.99:0/108355185 shutdown_connections 2024-05-14T17:16:18.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 -- 172.21.15.99:0/108355185 wait complete. 2024-05-14T17:16:18.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 Processor -- start 2024-05-14T17:16:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.617+0000 7f78b6673700 1 -- start start 2024-05-14T17:16:18.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b6673700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78b0102440 0x7f78b0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:18.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b6673700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:18.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b6673700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b0108520 0x7f78b0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:18.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b6673700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0112670 con 0x7f78b0102e00 2024-05-14T17:16:18.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b6673700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01124f0 con 0x7f78b0108520 2024-05-14T17:16:18.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b6673700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01127f0 con 0x7f78b0102440 2024-05-14T17:16:18.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:18.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35642/0 (socket says 172.21.15.99:35642) 2024-05-14T17:16:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 -- 172.21.15.99:0/2018195467 learned_addr learned my addr 172.21.15.99:0/2018195467 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:18.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78affff700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78b0102440 0x7f78b0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:19.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78b4c10700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b0108520 0x7f78b0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 -- 172.21.15.99:0/2018195467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78b0102440 msgr2=0x7f78b0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78b0102440 0x7f78b0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:19.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 -- 172.21.15.99:0/2018195467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b0108520 msgr2=0x7f78b0104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b0108520 0x7f78b0104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 -- 172.21.15.99:0/2018195467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b01a8230 con 0x7f78b0102e00 2024-05-14T17:16:19.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.618+0000 7f78af7fe700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b01045c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f789800d980 tx=0x7f789800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:19.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.619+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898017070 con 0x7f78b0102e00 2024-05-14T17:16:19.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.619+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898005020 con 0x7f78b0102e00 2024-05-14T17:16:19.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.619+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b01a8520 con 0x7f78b0102e00 2024-05-14T17:16:19.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.619+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789800fa10 con 0x7f78b0102e00 2024-05-14T17:16:19.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.619+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b01a8a60 con 0x7f78b0102e00 2024-05-14T17:16:19.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.620+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b006a230 con 0x7f78b0102e00 2024-05-14T17:16:19.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.621+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7898004a30 con 0x7f78b0102e00 2024-05-14T17:16:19.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.622+0000 7f78ad7fa700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f789c0b2a70 0x7f789c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:19.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.622+0000 7f78affff700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f789c0b2a70 0x7f789c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.622+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f78980e2850 con 0x7f78b0102e00 2024-05-14T17:16:19.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.622+0000 7f78affff700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f789c0b2a70 0x7f789c0b4f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009a10 tx=0x7f78a0009190 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:19.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.625+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7898020020 con 0x7f78b0102e00 2024-05-14T17:16:19.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.880+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78b0105560 con 0x7f78b0102e00 2024-05-14T17:16:19.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.881+0000 7f78ad7fa700 1 -- 172.21.15.99:0/2018195467 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11873 (secure 0 0 0) 0x7f78980aa2b0 con 0x7f78b0102e00 2024-05-14T17:16:19.274 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:16:19.280 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:15:44.595162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:16:19.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f789c0b2a70 msgr2=0x7f789c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:19.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f789c0b2a70 0x7f789c0b4f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009a10 tx=0x7f78a0009190 comp rx=0 tx=0).stop 2024-05-14T17:16:19.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 msgr2=0x7f78b01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:19.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b01045c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f789800d980 tx=0x7f789800dc90 comp rx=0 tx=0).stop 2024-05-14T17:16:19.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 shutdown_connections 2024-05-14T17:16:19.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f789c0b2a70 0x7f789c0b4f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:19.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b0108520 0x7f78b0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:19.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78b0102440 0x7f78b0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:19.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 --2- 172.21.15.99:0/2018195467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b0102e00 0x7f78b01045c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:19.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 >> 172.21.15.99:0/2018195467 conn(0x7f78b00793f0 msgr2=0x7f78b006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:19.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 shutdown_connections 2024-05-14T17:16:19.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:18.883+0000 7f78b6673700 1 -- 172.21.15.99:0/2018195467 wait complete. 2024-05-14T17:16:19.444 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-05-14T17:16:19.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:19 smithi150 ceph-mon[36204]: pgmap v399: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-05-14T17:16:19.457 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2018195467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:19.511 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:19 smithi099 ceph-mon[31851]: pgmap v399: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-05-14T17:16:19.517 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2018195467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:19.539 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --set-id 7f40df97 --release --await' 2024-05-14T17:16:19.545 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --set-id 7f40df97 --release --await 2024-05-14T17:16:19.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:19 smithi193 ceph-mon[35962]: pgmap v399: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-05-14T17:16:19.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2018195467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:16:19.599 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.76 MB/sec execute 511 sec latency 27704.575 ms 2024-05-14T17:16:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 -- 172.21.15.99:0/29431411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 msgr2=0x7f6b3810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/29431411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3810f4c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009ca0 tx=0x7f6b3002a9e0 comp rx=0 tx=0).stop 2024-05-14T17:16:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 -- 172.21.15.99:0/29431411 shutdown_connections 2024-05-14T17:16:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/29431411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3810f4c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/29431411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38104600 0x7f6b3810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/29431411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 0x7f6b381040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 -- 172.21.15.99:0/29431411 >> 172.21.15.99:0/29431411 conn(0x7f6b38078ad0 msgr2=0x7f6b38078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 -- 172.21.15.99:0/29431411 shutdown_connections 2024-05-14T17:16:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.531+0000 7f6b3fb78640 1 -- 172.21.15.99:0/29431411 wait complete. 2024-05-14T17:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 Processor -- start 2024-05-14T17:16:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 -- start start 2024-05-14T17:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 0x7f6b38199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38104600 0x7f6b38199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38112cd0 con 0x7f6b3810d020 2024-05-14T17:16:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38112b50 con 0x7f6b38103ce0 2024-05-14T17:16:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3fb78640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38112e50 con 0x7f6b38104600 2024-05-14T17:16:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35662/0 (socket says 172.21.15.99:35662) 2024-05-14T17:16:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 -- 172.21.15.99:0/76674022 learned_addr learned my addr 172.21.15.99:0/76674022 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3d113640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38104600 0x7f6b38199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3d914640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 0x7f6b38199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 -- 172.21.15.99:0/76674022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38104600 msgr2=0x7f6b38199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38104600 0x7f6b38199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 -- 172.21.15.99:0/76674022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 msgr2=0x7f6b38199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 0x7f6b38199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3e115640 1 -- 172.21.15.99:0/76674022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3819e0b0 con 0x7f6b3810d020 2024-05-14T17:16:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.532+0000 7f6b3d914640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 0x7f6b38199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.533+0000 7f6b3e115640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3819d980 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009dc0 tx=0x7f6b3002e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.533+0000 7f6b2effd640 1 -- 172.21.15.99:0/76674022 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30033040 con 0x7f6b3810d020 2024-05-14T17:16:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.533+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3819e340 con 0x7f6b3810d020 2024-05-14T17:16:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.533+0000 7f6b2effd640 1 -- 172.21.15.99:0/76674022 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b30038b00 con 0x7f6b3810d020 2024-05-14T17:16:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.533+0000 7f6b2effd640 1 -- 172.21.15.99:0/76674022 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30043900 con 0x7f6b3810d020 2024-05-14T17:16:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.533+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b381a5c30 con 0x7f6b3810d020 2024-05-14T17:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.535+0000 7f6b4296a740 1 Processor -- start 2024-05-14T17:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.536+0000 7f6b2effd640 1 -- 172.21.15.99:0/76674022 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6b30036070 con 0x7f6b3810d020 2024-05-14T17:16:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 -- start start 2024-05-14T17:16:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa6f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f6130 0x5579aa7049a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa7072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5579aa668c30 con 0x5579aa6f5810 2024-05-14T17:16:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5579aa6587d0 con 0x5579aa6f6130 2024-05-14T17:16:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b4296a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5579aa65c810 con 0x5579aa704ee0 2024-05-14T17:16:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa6f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa6f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35666/0 (socket says 172.21.15.99:35666) 2024-05-14T17:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 -- 172.21.15.99:0/798901332 learned_addr learned my addr 172.21.15.99:0/798901332 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d914640 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f6130 0x5579aa7049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b2effd640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b240b2a10 0x7f6b240b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 -- 172.21.15.99:0/798901332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 msgr2=0x5579aa7072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa7072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 -- 172.21.15.99:0/798901332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f6130 msgr2=0x5579aa7049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f6130 0x5579aa7049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d113640 1 -- 172.21.15.99:0/798901332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579aa36d210 con 0x5579aa6f5810 2024-05-14T17:16:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3d914640 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f6130 0x5579aa7049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.537+0000 7f6b3e115640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b240b2a10 0x7f6b240b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.538+0000 7f6b3d113640 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa6f5bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ac90 tx=0x7f6b280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5535eadd38c69a88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.538+0000 7f6b2effd640 1 -- 172.21.15.99:0/76674022 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f6b30103660 con 0x7f6b3810d020 2024-05-14T17:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.538+0000 7f6b23fff640 1 -- 172.21.15.99:0/798901332 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28012450 con 0x5579aa6f5810 2024-05-14T17:16:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.538+0000 7f6b23fff640 1 -- 172.21.15.99:0/798901332 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28012aa0 con 0x5579aa6f5810 2024-05-14T17:16:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.538+0000 7f6b23fff640 1 -- 172.21.15.99:0/798901332 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800e8d0 con 0x5579aa6f5810 2024-05-14T17:16:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- 172.21.15.99:0/798901332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 msgr2=0x5579aa6f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b3e115640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b240b2a10 0x7f6b240b4ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b300c7bb0 tx=0x7f6b300ed040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa6f5bf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ac90 tx=0x7f6b280117b0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- 172.21.15.99:0/798901332 shutdown_connections 2024-05-14T17:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa7072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f6130 0x5579aa7049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- 172.21.15.99:0/798901332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa6f5bf0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- 172.21.15.99:0/798901332 >> 172.21.15.99:0/798901332 conn(0x5579aa6f1e00 msgr2=0x5579aa6f21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- 172.21.15.99:0/798901332 shutdown_connections 2024-05-14T17:16:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- 172.21.15.99:0/798901332 wait complete. 2024-05-14T17:16:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- start start 2024-05-14T17:16:21.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 Processor -- start 2024-05-14T17:16:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa704900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa701bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa704ee0 0x5579aa702200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5579aa33e790 con 0x5579aa6f5810 2024-05-14T17:16:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.539+0000 7f6b4296a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5579aa614890 con 0x5579aa704ee0 2024-05-14T17:16:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b4296a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5579aa707bb0 con 0x5579aa6f6130 2024-05-14T17:16:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b4296a740 10 client.0 _ll_register_callbacks cb 0x5579aa617d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa704900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa704900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35680/0 (socket says 172.21.15.99:35680) 2024-05-14T17:16:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 -- 172.21.15.99:0/3414229711 learned_addr learned my addr 172.21.15.99:0/3414229711 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d914640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa701bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 -- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 msgr2=0x5579aa701bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa701bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 -- 172.21.15.99:0/3414229711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa704ee0 msgr2=0x5579aa702200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa704ee0 0x5579aa702200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 -- 172.21.15.99:0/3414229711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579aa439620 con 0x5579aa6f5810 2024-05-14T17:16:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d914640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa701bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b3d113640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa704900 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6b28012f60 tx=0x7f6b2800aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42072d26c22f0af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b217fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b217fa640 1 -- 172.21.15.99:0/3414229711 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28005040 con 0x5579aa6f5810 2024-05-14T17:16:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b217fa640 1 -- 172.21.15.99:0/3414229711 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f6b280051e0 con 0x5579aa6f5810 2024-05-14T17:16:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b28012aa0 con 0x5579aa6f5810 2024-05-14T17:16:21.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b217fa640 1 -- 172.21.15.99:0/3414229711 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6b280055f0 con 0x5579aa6f5810 2024-05-14T17:16:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b4296a740 10 client.16848 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.540+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5579aa713640 con 0x5579aa6f5810 2024-05-14T17:16:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b217fa640 1 -- 172.21.15.99:0/3414229711 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28012aa0 con 0x5579aa6f5810 2024-05-14T17:16:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b217fa640 1 -- 172.21.15.99:0/3414229711 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f6b2800ea40 con 0x5579aa6f5810 2024-05-14T17:16:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b4296a740 10 client.16848 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:21.673 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b4296a740 10 client.16848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b4296a740 20 client.16848 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b4296a740 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa714a50 0x5579aa716e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b4296a740 4 client.16848 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:16:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579aa66ab80 con 0x5579aa714a50 2024-05-14T17:16:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.541+0000 7f6b3d914640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa714a50 0x5579aa716e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.542+0000 7f6b217fa640 10 client.16848 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.542+0000 7f6b3d914640 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa714a50 0x5579aa716e40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4cf645ee452671e server_cookie=c9a45f07e627efa3 in_seq=0 out_seq=0 2024-05-14T17:16:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.542+0000 7f6b217fa640 1 -- 172.21.15.99:0/3414229711 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579aa66ab80 con 0x5579aa714a50 2024-05-14T17:16:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.542+0000 7f6b217fa640 10 client.16848 handle_command_reply: tid=0 2024-05-14T17:16:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.542+0000 7f6b4296a740 1 client.16848 shutdown 2024-05-14T17:16:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 msgr2=0x5579aa704900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa704900 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6b28012f60 tx=0x7f6b2800aa50 comp rx=0 tx=0).stop 2024-05-14T17:16:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 shutdown_connections 2024-05-14T17:16:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa714a50 0x5579aa716e40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa704ee0 0x5579aa702200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa701bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 --2- 172.21.15.99:0/3414229711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f5810 0x5579aa704900 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b217fa640 0 client.16848 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 >> 172.21.15.99:0/3414229711 conn(0x5579aa6f1e00 msgr2=0x5579aa6f7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 shutdown_connections 2024-05-14T17:16:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 1 -- 172.21.15.99:0/3414229711 wait complete. 2024-05-14T17:16:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.543+0000 7f6b4296a740 20 client.16848 trim_cache size 0 max 16384 2024-05-14T17:16:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 Processor -- start 2024-05-14T17:16:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 -- start start 2024-05-14T17:16:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa710a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa704430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 0x5579aa71c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381aec90 con 0x5579aa704ee0 2024-05-14T17:16:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381ac560 con 0x5579aa6f5810 2024-05-14T17:16:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b4296a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381aa3d0 con 0x5579aa6f6130 2024-05-14T17:16:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 0x5579aa71c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 0x5579aa71c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35694/0 (socket says 172.21.15.99:35694) 2024-05-14T17:16:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 -- 172.21.15.99:0/1108856710 learned_addr learned my addr 172.21.15.99:0/1108856710 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d914640 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa704430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa710a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 -- 172.21.15.99:0/1108856710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 msgr2=0x5579aa704430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa704430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 -- 172.21.15.99:0/1108856710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 msgr2=0x5579aa710a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa710a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.558+0000 7f6b3d113640 1 -- 172.21.15.99:0/1108856710 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579aa700140 con 0x5579aa704ee0 2024-05-14T17:16:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b3d113640 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 0x5579aa71c600 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6b28001eb0 tx=0x7f6b2800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd77dfae9c210bb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b237fe640 1 -- 172.21.15.99:0/1108856710 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28012660 con 0x5579aa704ee0 2024-05-14T17:16:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b237fe640 1 -- 172.21.15.99:0/1108856710 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28022070 con 0x5579aa704ee0 2024-05-14T17:16:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b237fe640 1 -- 172.21.15.99:0/1108856710 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800f9a0 con 0x5579aa704ee0 2024-05-14T17:16:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 -- 172.21.15.99:0/1108856710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 msgr2=0x5579aa71c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 0x5579aa71c600 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6b28001eb0 tx=0x7f6b2800af90 comp rx=0 tx=0).stop 2024-05-14T17:16:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 -- 172.21.15.99:0/1108856710 shutdown_connections 2024-05-14T17:16:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa704ee0 0x5579aa71c600 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa6f6130 0x5579aa704430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 --2- 172.21.15.99:0/1108856710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa710a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 -- 172.21.15.99:0/1108856710 >> 172.21.15.99:0/1108856710 conn(0x5579aa6f1e00 msgr2=0x5579aa6fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 -- 172.21.15.99:0/1108856710 shutdown_connections 2024-05-14T17:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.559+0000 7f6b4296a740 1 -- 172.21.15.99:0/1108856710 wait complete. 2024-05-14T17:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 -- start start 2024-05-14T17:16:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 Processor -- start 2024-05-14T17:16:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa70d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 0x5579aa708540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa708a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381aaa20 con 0x5579aa6f6130 2024-05-14T17:16:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381abe90 con 0x5579aa6f5810 2024-05-14T17:16:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381ab0f0 con 0x5579aa704ee0 2024-05-14T17:16:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b4296a740 10 client.0 _ll_register_callbacks cb 0x7f6b2800a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b3d914640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 0x5579aa708540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b3d914640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 0x5579aa708540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35700/0 (socket says 172.21.15.99:35700) 2024-05-14T17:16:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b3d914640 1 -- 172.21.15.99:0/2253139878 learned_addr learned my addr 172.21.15.99:0/2253139878 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.560+0000 7f6b3d113640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa708a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d914640 1 -- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 msgr2=0x5579aa708a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d914640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa708a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d113640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa70d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d914640 1 -- 172.21.15.99:0/2253139878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 msgr2=0x5579aa70d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d914640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa70d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d914640 1 -- 172.21.15.99:0/2253139878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579aa7090f0 con 0x5579aa6f6130 2024-05-14T17:16:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d113640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa70d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b3d914640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 0x5579aa708540 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400ad10 tx=0x7f6b34012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9797b263f22523f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b217fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b217fa640 1 -- 172.21.15.99:0/2253139878 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34004220 con 0x5579aa6f6130 2024-05-14T17:16:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b28022070 con 0x5579aa6f6130 2024-05-14T17:16:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b217fa640 1 -- 172.21.15.99:0/2253139878 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f6b340043c0 con 0x5579aa6f6130 2024-05-14T17:16:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b217fa640 1 -- 172.21.15.99:0/2253139878 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6b34012e50 con 0x5579aa6f6130 2024-05-14T17:16:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 10 client.16854 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5579aa77f0e0 con 0x5579aa6f6130 2024-05-14T17:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b217fa640 1 -- 172.21.15.99:0/2253139878 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28022070 con 0x5579aa6f6130 2024-05-14T17:16:23.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b217fa640 1 -- 172.21.15.99:0/2253139878 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f6b340035f0 con 0x5579aa6f6130 2024-05-14T17:16:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 10 client.16854 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 20 client.16854 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa77f840 0x5579aa781c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 4 client.16854 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:16:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.561+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579aa664360 con 0x5579aa77f840 2024-05-14T17:16:23.125 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.75 MB/sec execute 512 sec latency 28704.601 ms 2024-05-14T17:16:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.562+0000 7f6b3d113640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa77f840 0x5579aa781c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.562+0000 7f6b217fa640 10 client.16854 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.562+0000 7f6b3d113640 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa77f840 0x5579aa781c60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe77056a7630e770 server_cookie=c6d8ee93fcce0f3d in_seq=0 out_seq=0 2024-05-14T17:16:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b217fa640 1 -- 172.21.15.99:0/2253139878 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5579aa664360 con 0x5579aa77f840 2024-05-14T17:16:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b217fa640 10 client.16854 handle_command_reply: tid=0 2024-05-14T17:16:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 client.16854 shutdown 2024-05-14T17:16:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 msgr2=0x5579aa708540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 0x5579aa708540 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400ad10 tx=0x7f6b34012b10 comp rx=0 tx=0).stop 2024-05-14T17:16:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 shutdown_connections 2024-05-14T17:16:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5579aa77f840 0x5579aa781c60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579aa704ee0 0x5579aa708a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579aa6f6130 0x5579aa708540 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b217fa640 0 client.16854 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 --2- 172.21.15.99:0/2253139878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5579aa6f5810 0x5579aa70d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 >> 172.21.15.99:0/2253139878 conn(0x5579aa6f1e00 msgr2=0x5579aa6fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 shutdown_connections 2024-05-14T17:16:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.585+0000 7f6b4296a740 1 -- 172.21.15.99:0/2253139878 wait complete. 2024-05-14T17:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b4296a740 20 client.16854 trim_cache size 0 max 16384 2024-05-14T17:16:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:16:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:16:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:16:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b240b2a10 msgr2=0x7f6b240b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b240b2a10 0x7f6b240b4ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6b300c7bb0 tx=0x7f6b300ed040 comp rx=0 tx=0).stop 2024-05-14T17:16:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 msgr2=0x7f6b3819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3819d980 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009dc0 tx=0x7f6b3002e5c0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 shutdown_connections 2024-05-14T17:16:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f6b240b2a10 0x7f6b240b4ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b3810d020 0x7f6b3819d980 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38104600 0x7f6b38199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 --2- 172.21.15.99:0/76674022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b38103ce0 0x7f6b38199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 >> 172.21.15.99:0/76674022 conn(0x7f6b38078ad0 msgr2=0x7f6b380fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 shutdown_connections 2024-05-14T17:16:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:20.586+0000 7f6b3fb78640 1 -- 172.21.15.99:0/76674022 wait complete. 2024-05-14T17:16:23.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:21 smithi099 ceph-mon[31851]: pgmap v400: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 18 op/s 2024-05-14T17:16:23.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:21 smithi193 ceph-mon[35962]: pgmap v400: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 18 op/s 2024-05-14T17:16:23.817 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:23 smithi193 ceph-mon[35962]: pgmap v401: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:16:23.829 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:21 smithi150 ceph-mon[36204]: pgmap v400: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 18 op/s 2024-05-14T17:16:23.842 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:23 smithi150 ceph-mon[36204]: pgmap v401: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:16:23.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 -- 172.21.15.99:0/1553227819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102470 msgr2=0x7f70681028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:23.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 --2- 172.21.15.99:0/1553227819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102470 0x7f70681028f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7054009990 tx=0x7f7054009ca0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 -- 172.21.15.99:0/1553227819 shutdown_connections 2024-05-14T17:16:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 --2- 172.21.15.99:0/1553227819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068102e30 0x7f706810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 --2- 172.21.15.99:0/1553227819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70681084b0 0x7f7068108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 --2- 172.21.15.99:0/1553227819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102470 0x7f70681028f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 -- 172.21.15.99:0/1553227819 >> 172.21.15.99:0/1553227819 conn(0x7f70680793f0 msgr2=0x7f7068079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:23.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 -- 172.21.15.99:0/1553227819 shutdown_connections 2024-05-14T17:16:24.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.526+0000 7f706efa4700 1 -- 172.21.15.99:0/1553227819 wait complete. 2024-05-14T17:16:24.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.527+0000 7f706efa4700 1 Processor -- start 2024-05-14T17:16:24.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.527+0000 7f706efa4700 1 -- start start 2024-05-14T17:16:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068102470 0x7f7068104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 0x7f70681045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70681084b0 0x7f7068104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7068112660 con 0x7f7068102e30 2024-05-14T17:16:24.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70681124e0 con 0x7f70681084b0 2024-05-14T17:16:24.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70681127e0 con 0x7f7068102470 2024-05-14T17:16:24.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 0x7f70681045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 0x7f70681045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35710/0 (socket says 172.21.15.99:35710) 2024-05-14T17:16:24.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 -- 172.21.15.99:0/1592611411 learned_addr learned my addr 172.21.15.99:0/1592611411 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:24.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706cd40700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068102470 0x7f7068104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706d541700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70681084b0 0x7f7068104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 -- 172.21.15.99:0/1592611411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068102470 msgr2=0x7f7068104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:24.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068102470 0x7f7068104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 -- 172.21.15.99:0/1592611411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70681084b0 msgr2=0x7f7068104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:24.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70681084b0 0x7f7068104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 -- 172.21.15.99:0/1592611411 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70681a81d0 con 0x7f7068102e30 2024-05-14T17:16:24.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7067fff700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 0x7f70681045c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f705400b5c0 tx=0x7f70540058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7065ffb700 1 -- 172.21.15.99:0/1592611411 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054027070 con 0x7f7068102e30 2024-05-14T17:16:24.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7065ffb700 1 -- 172.21.15.99:0/1592611411 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7054003fb0 con 0x7f7068102e30 2024-05-14T17:16:24.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70681a8460 con 0x7f7068102e30 2024-05-14T17:16:24.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.528+0000 7f7065ffb700 1 -- 172.21.15.99:0/1592611411 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054022ce0 con 0x7f7068102e30 2024-05-14T17:16:24.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.529+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70681a88b0 con 0x7f7068102e30 2024-05-14T17:16:24.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.531+0000 7f7065ffb700 1 -- 172.21.15.99:0/1592611411 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f705402a070 con 0x7f7068102e30 2024-05-14T17:16:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.532+0000 7f7065ffb700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f70580b29f0 0x7f70580b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.532+0000 7f706cd40700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f70580b29f0 0x7f70580b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.532+0000 7f7065ffb700 1 -- 172.21.15.99:0/1592611411 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f70540e35a0 con 0x7f7068102e30 2024-05-14T17:16:24.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 Processor -- start 2024-05-14T17:16:24.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 -- start start 2024-05-14T17:16:24.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f706cd40700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f70580b29f0 0x7f70580b4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f705c009a10 tx=0x7f705c009190 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:24.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e1e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ebd1d76d90 con 0x55ebd1e1e6c0 2024-05-14T17:16:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ebd1d66930 con 0x55ebd1e1dda0 2024-05-14T17:16:24.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f707983ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ebd1d6a970 con 0x55ebd1e18ad0 2024-05-14T17:16:24.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e184b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35718/0 (socket says 172.21.15.99:35718) 2024-05-14T17:16:24.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 -- 172.21.15.99:0/1025241489 learned_addr learned my addr 172.21.15.99:0/1025241489 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:24.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f706cd40700 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e1e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f706d541700 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 -- 172.21.15.99:0/1025241489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e18ad0 msgr2=0x55ebd1e18f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:24.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e18f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 -- 172.21.15.99:0/1025241489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1dda0 msgr2=0x55ebd1e1e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:24.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e1e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 -- 172.21.15.99:0/1025241489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd1cb69d0 con 0x55ebd1e1e6c0 2024-05-14T17:16:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.533+0000 7f7067fff700 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e184b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7054006010 tx=0x7f70540a68a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850820101cc3c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:24.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f7052ffd700 1 -- 172.21.15.99:0/1025241489 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540aa3c0 con 0x55ebd1e1e6c0 2024-05-14T17:16:24.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f7052ffd700 1 -- 172.21.15.99:0/1025241489 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70540aafb0 con 0x55ebd1e1e6c0 2024-05-14T17:16:24.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f7052ffd700 1 -- 172.21.15.99:0/1025241489 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540b10b0 con 0x55ebd1e1e6c0 2024-05-14T17:16:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 -- 172.21.15.99:0/1025241489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 msgr2=0x55ebd1e184b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:24.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e184b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7054006010 tx=0x7f70540a68a0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 -- 172.21.15.99:0/1025241489 shutdown_connections 2024-05-14T17:16:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e18f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e1e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 --2- 172.21.15.99:0/1025241489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e184b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:24.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 -- 172.21.15.99:0/1025241489 >> 172.21.15.99:0/1025241489 conn(0x55ebd1d43350 msgr2=0x55ebd1e12ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.534+0000 7f707983ab80 1 -- 172.21.15.99:0/1025241489 shutdown_connections 2024-05-14T17:16:24.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 -- 172.21.15.99:0/1025241489 wait complete. 2024-05-14T17:16:24.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 -- start start 2024-05-14T17:16:24.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 Processor -- start 2024-05-14T17:16:24.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e21d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:24.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ebd1c6d0a0 con 0x55ebd1e1e6c0 2024-05-14T17:16:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ebd1d088c0 con 0x55ebd1e18ad0 2024-05-14T17:16:24.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ebd1d76d90 con 0x55ebd1e1dda0 2024-05-14T17:16:24.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f707983ab80 10 client.0 _ll_register_callbacks cb 0x55ebd1cfd680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:24.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f706d541700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e21d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:24.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f706d541700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e21d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35726/0 (socket says 172.21.15.99:35726) 2024-05-14T17:16:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.535+0000 7f706d541700 1 -- 172.21.15.99:0/241269725 learned_addr learned my addr 172.21.15.99:0/241269725 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:25.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706cd40700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:25.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f7067fff700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:25.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706d541700 1 -- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 msgr2=0x55ebd1e217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:25.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706d541700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706d541700 1 -- 172.21.15.99:0/241269725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 msgr2=0x55ebd1e24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:25.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706d541700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e24430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706d541700 1 -- 172.21.15.99:0/241269725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd1d05790 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f706d541700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e21d10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7060007900 tx=0x7f7060007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3164eb576b9a83c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f7047fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f7047fff700 1 -- 172.21.15.99:0/241269725 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060016660 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f7047fff700 1 -- 172.21.15.99:0/241269725 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f7060016800 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70540aafb0 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f7047fff700 1 -- 172.21.15.99:0/241269725 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7060024050 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f707983ab80 10 client.16866 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:25.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ebd1e2f4b0 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.536+0000 7f7047fff700 1 -- 172.21.15.99:0/241269725 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70540aafb0 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f7047fff700 1 -- 172.21.15.99:0/241269725 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f7060011c60 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f707983ab80 10 client.16866 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:25.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f707983ab80 10 client.16866 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:16:25.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f707983ab80 20 client.16866 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:25.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f707983ab80 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1e31af0 0x55ebd1e33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:25.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f707983ab80 4 client.16866 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:16:25.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebd1de4460 con 0x55ebd1e31af0 2024-05-14T17:16:25.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f706cd40700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1e31af0 0x55ebd1e33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:25.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f706cd40700 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1e31af0 0x55ebd1e33f40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bdcadbf5ab48c45 server_cookie=5f203930fc9893e in_seq=0 out_seq=0 2024-05-14T17:16:25.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.537+0000 7f7047fff700 10 client.16866 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f7047fff700 1 -- 172.21.15.99:0/241269725 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ebd1de4460 con 0x55ebd1e31af0 2024-05-14T17:16:25.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f7047fff700 10 client.16866 handle_command_reply: tid=0 2024-05-14T17:16:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 client.16866 shutdown 2024-05-14T17:16:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 msgr2=0x55ebd1e21d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:25.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e21d10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7060007900 tx=0x7f7060007c10 comp rx=0 tx=0).stop 2024-05-14T17:16:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 shutdown_connections 2024-05-14T17:16:25.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1e31af0 0x55ebd1e33f40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e21d10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 --2- 172.21.15.99:0/241269725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f7047fff700 0 client.16866 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:25.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 >> 172.21.15.99:0/241269725 conn(0x55ebd1d43350 msgr2=0x55ebd1e0d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 shutdown_connections 2024-05-14T17:16:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.538+0000 7f707983ab80 1 -- 172.21.15.99:0/241269725 wait complete. 2024-05-14T17:16:25.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.539+0000 7f707983ab80 20 client.16866 trim_cache size 0 max 16384 2024-05-14T17:16:25.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 Processor -- start 2024-05-14T17:16:25.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 -- start start 2024-05-14T17:16:25.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:25.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e23cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:25.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70681b2760 con 0x55ebd1e1dda0 2024-05-14T17:16:25.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70681afff0 con 0x55ebd1e18ad0 2024-05-14T17:16:25.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f707983ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70681ade60 con 0x55ebd1e1e6c0 2024-05-14T17:16:25.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e23cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:25.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e23cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35730/0 (socket says 172.21.15.99:35730) 2024-05-14T17:16:25.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 -- 172.21.15.99:0/1122512000 learned_addr learned my addr 172.21.15.99:0/1122512000 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:25.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f706d541700 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:25.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f706cd40700 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:25.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 -- 172.21.15.99:0/1122512000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1e6c0 msgr2=0x55ebd1e210d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:25.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e210d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 -- 172.21.15.99:0/1122512000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 msgr2=0x55ebd1e184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:25.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e184b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 -- 172.21.15.99:0/1122512000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebd1e1e3d0 con 0x55ebd1e1dda0 2024-05-14T17:16:25.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.558+0000 7f7067fff700 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e23cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f70540a61b0 tx=0x7f70540a6770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb53df6191914a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:25.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f70527fc700 1 -- 172.21.15.99:0/1122512000 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540aafb0 con 0x55ebd1e1dda0 2024-05-14T17:16:25.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f70527fc700 1 -- 172.21.15.99:0/1122512000 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70540aec60 con 0x55ebd1e1dda0 2024-05-14T17:16:25.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f70527fc700 1 -- 172.21.15.99:0/1122512000 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054004160 con 0x55ebd1e1dda0 2024-05-14T17:16:25.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 -- 172.21.15.99:0/1122512000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 msgr2=0x55ebd1e23cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:25.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e23cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f70540a61b0 tx=0x7f70540a6770 comp rx=0 tx=0).stop 2024-05-14T17:16:25.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 -- 172.21.15.99:0/1122512000 shutdown_connections 2024-05-14T17:16:25.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 --2- 172.21.15.99:0/1122512000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e23cd0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:25.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 -- 172.21.15.99:0/1122512000 >> 172.21.15.99:0/1122512000 conn(0x55ebd1d43350 msgr2=0x55ebd1e235c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:25.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 -- 172.21.15.99:0/1122512000 shutdown_connections 2024-05-14T17:16:26.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.559+0000 7f707983ab80 1 -- 172.21.15.99:0/1122512000 wait complete. 2024-05-14T17:16:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 -- start start 2024-05-14T17:16:26.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 Processor -- start 2024-05-14T17:16:26.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:26.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e279e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:26.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70681ae4b0 con 0x55ebd1e18ad0 2024-05-14T17:16:26.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70681af920 con 0x55ebd1e1e6c0 2024-05-14T17:16:26.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70681aeb80 con 0x55ebd1e1dda0 2024-05-14T17:16:26.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f707983ab80 10 client.0 _ll_register_callbacks cb 0x55ebd1cfd680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:26.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e26f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35732/0 (socket says 172.21.15.99:35732) 2024-05-14T17:16:26.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 -- 172.21.15.99:0/2348727153 learned_addr learned my addr 172.21.15.99:0/2348727153 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f7067fff700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:26.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706d541700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e279e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:26.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 -- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 msgr2=0x55ebd1e274a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:26.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e274a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 -- 172.21.15.99:0/2348727153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1e6c0 msgr2=0x55ebd1e279e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.560+0000 7f706cd40700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e279e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:26.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f706cd40700 1 -- 172.21.15.99:0/2348727153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebd1ea4be0 con 0x55ebd1e18ad0 2024-05-14T17:16:26.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f706d541700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:26.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f706cd40700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e26f60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f705c00e4b0 tx=0x7f705c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d83dc2d41cab6a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:26.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f7047fff700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:26.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f7047fff700 1 -- 172.21.15.99:0/2348727153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c02d070 con 0x55ebd1e18ad0 2024-05-14T17:16:26.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f7047fff700 1 -- 172.21.15.99:0/2348727153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f705c015890 con 0x55ebd1e18ad0 2024-05-14T17:16:26.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebd1d78ce0 con 0x55ebd1e18ad0 2024-05-14T17:16:26.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f7047fff700 1 -- 172.21.15.99:0/2348727153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f705c036050 con 0x55ebd1e18ad0 2024-05-14T17:16:26.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f707983ab80 10 client.16872 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:26.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ebd1ea5ec0 con 0x55ebd1e18ad0 2024-05-14T17:16:26.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f7047fff700 1 -- 172.21.15.99:0/2348727153 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ebd1d78ce0 con 0x55ebd1e18ad0 2024-05-14T17:16:26.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f7047fff700 1 -- 172.21.15.99:0/2348727153 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f705c030020 con 0x55ebd1e18ad0 2024-05-14T17:16:26.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f707983ab80 10 client.16872 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:26.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f707983ab80 10 client.16872 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:16:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.561+0000 7f707983ab80 20 client.16872 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.562+0000 7f707983ab80 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1ea6600 0x55ebd1ea8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:26.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.562+0000 7f707983ab80 4 client.16872 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7f40df97", "release": true, "await": true}] 2024-05-14T17:16:26.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.562+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7f40df97", "release": true, "await": true}) -- 0x55ebd1d724c0 con 0x55ebd1ea6600 2024-05-14T17:16:26.511 INFO:tasks.workunit.client.0.smithi099.stdout: 1 492072 3.74 MB/sec execute 513 sec latency 29704.629 ms 2024-05-14T17:16:26.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.562+0000 7f7067fff700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1ea6600 0x55ebd1ea8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:26.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.562+0000 7f7067fff700 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1ea6600 0x55ebd1ea8a20 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd358f1519f0e58 server_cookie=877aacb9efdb7247 in_seq=0 out_seq=0 2024-05-14T17:16:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.562+0000 7f7047fff700 10 client.16872 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f7047fff700 1 -- 172.21.15.99:0/2348727153 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ebd1d724c0 con 0x55ebd1ea6600 2024-05-14T17:16:26.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f7047fff700 10 client.16872 handle_command_reply: tid=0 2024-05-14T17:16:26.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 client.16872 shutdown 2024-05-14T17:16:26.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 msgr2=0x55ebd1e26f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:26.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e26f60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f705c00e4b0 tx=0x7f705c011000 comp rx=0 tx=0).stop 2024-05-14T17:16:26.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 shutdown_connections 2024-05-14T17:16:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ebd1e1e6c0 0x55ebd1e279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:26.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55ebd1ea6600 0x55ebd1ea8a20 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:26.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebd1e18ad0 0x55ebd1e26f60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 --2- 172.21.15.99:0/2348727153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebd1e1dda0 0x55ebd1e274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:26.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 >> 172.21.15.99:0/2348727153 conn(0x55ebd1d43350 msgr2=0x55ebd1e200c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:26.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f7047fff700 0 client.16872 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:26.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 shutdown_connections 2024-05-14T17:16:26.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 1 -- 172.21.15.99:0/2348727153 wait complete. 2024-05-14T17:16:26.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.568+0000 7f707983ab80 20 client.16872 trim_cache size 0 max 16384 2024-05-14T17:16:26.699 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:16:26.706 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 42, 2024-05-14T17:16:26.713 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:16:26.724 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 44, 2024-05-14T17:16:26.745 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:16:26.791 INFO:teuthology.orchestra.run.smithi099.stdout: "7f40df97": { 2024-05-14T17:16:26.807 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 44, 2024-05-14T17:16:26.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:16:26.823 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:16:26.830 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:16:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:16:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:16:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:16:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 149.7, 2024-05-14T17:16:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:16:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:16:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:16:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:16:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RELEASED", 2024-05-14T17:16:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:16:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:16:26.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:16:26.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:16:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:16:26.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:16:26.999 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:16:27.008 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:25 smithi193 ceph-mon[35962]: pgmap v402: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 412 KiB/s wr, 54 op/s 2024-05-14T17:16:27.015 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:25 smithi150 ceph-mon[36204]: pgmap v402: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 412 KiB/s wr, 54 op/s 2024-05-14T17:16:27.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f70580b29f0 msgr2=0x7f70580b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:27.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f70580b29f0 0x7f70580b4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f705c009a10 tx=0x7f705c009190 comp rx=0 tx=0).stop 2024-05-14T17:16:27.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 msgr2=0x7f70681045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:27.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 0x7f70681045c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f705400b5c0 tx=0x7f70540058e0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 shutdown_connections 2024-05-14T17:16:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f70580b29f0 0x7f70580b4eb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70681084b0 0x7f7068104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068102470 0x7f7068104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 --2- 172.21.15.99:0/1592611411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7068102e30 0x7f70681045c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 >> 172.21.15.99:0/1592611411 conn(0x7f70680793f0 msgr2=0x7f706806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 shutdown_connections 2024-05-14T17:16:27.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:16:21.569+0000 7f706efa4700 1 -- 172.21.15.99:0/1592611411 wait complete. 2024-05-14T17:16:27.144 INFO:tasks.workunit.client.0.smithi099.stdout: 1 494262 3.76 MB/sec execute 514 sec latency 29714.152 ms 2024-05-14T17:16:27.150 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:23 smithi099 ceph-mon[31851]: pgmap v401: 97 pgs: 97 active+clean; 138 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-05-14T17:16:27.159 INFO:tasks.workunit.client.0.smithi099.stdout: 1 495544 3.76 MB/sec execute 515 sec latency 107.276 ms 2024-05-14T17:16:27.165 INFO:tasks.workunit.client.0.smithi099.stdout: 1 495544 3.76 MB/sec execute 516 sec latency 1107.313 ms 2024-05-14T17:16:27.171 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:25 smithi099 ceph-mon[31851]: pgmap v402: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 412 KiB/s wr, 54 op/s 2024-05-14T17:16:27.202 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7f40df97', total seconds elapsed: 40.3 2024-05-14T17:16:27.208 INFO:tasks.workunit.client.0.smithi099.stdout: 1 497766 3.78 MB/sec execute 517 sec latency 1149.301 ms 2024-05-14T17:16:27.214 INFO:tasks.workunit.client.0.smithi099.stdout: 1 499092 3.78 MB/sec execute 518 sec latency 74.288 ms 2024-05-14T17:16:27.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:27 smithi150 ceph-mon[36204]: pgmap v403: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 380 KiB/s wr, 49 op/s 2024-05-14T17:16:27.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:27 smithi099 ceph-mon[31851]: pgmap v403: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 380 KiB/s wr, 49 op/s 2024-05-14T17:16:27.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:27 smithi193 ceph-mon[35962]: pgmap v403: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 380 KiB/s wr, 49 op/s 2024-05-14T17:16:27.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 499092 3.77 MB/sec execute 519 sec latency 1074.380 ms 2024-05-14T17:16:28.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 499092 3.76 MB/sec execute 520 sec latency 2074.480 ms 2024-05-14T17:16:29.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:29 smithi150 ceph-mon[36204]: pgmap v404: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 344 KiB/s wr, 45 op/s 2024-05-14T17:16:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:29 smithi099 ceph-mon[31851]: pgmap v404: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 344 KiB/s wr, 45 op/s 2024-05-14T17:16:29.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:29 smithi193 ceph-mon[35962]: pgmap v404: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 344 KiB/s wr, 45 op/s 2024-05-14T17:16:29.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 499092 3.76 MB/sec execute 521 sec latency 3074.546 ms 2024-05-14T17:16:30.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 501353 3.78 MB/sec execute 522 sec latency 3083.124 ms 2024-05-14T17:16:31.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:31 smithi150 ceph-mon[36204]: pgmap v405: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 660 KiB/s wr, 82 op/s 2024-05-14T17:16:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:31 smithi099 ceph-mon[31851]: pgmap v405: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 660 KiB/s wr, 82 op/s 2024-05-14T17:16:31.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:31 smithi193 ceph-mon[35962]: pgmap v405: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 660 KiB/s wr, 82 op/s 2024-05-14T17:16:31.561 INFO:tasks.workunit.client.0.smithi099.stdout: 1 502633 3.78 MB/sec execute 523 sec latency 137.793 ms 2024-05-14T17:16:32.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 502633 3.77 MB/sec execute 524 sec latency 1137.889 ms 2024-05-14T17:16:33.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:33 smithi150 ceph-mon[36204]: pgmap v406: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 646 KiB/s wr, 77 op/s 2024-05-14T17:16:33.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:33 smithi099 ceph-mon[31851]: pgmap v406: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 646 KiB/s wr, 77 op/s 2024-05-14T17:16:33.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:33 smithi193 ceph-mon[35962]: pgmap v406: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 646 KiB/s wr, 77 op/s 2024-05-14T17:16:33.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 502633 3.76 MB/sec execute 525 sec latency 2137.993 ms 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 -- 172.21.15.99:0/3827417563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c103940 msgr2=0x7f119c06b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3827417563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c103940 0x7f119c06b2c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1198009900 tx=0x7f119802a410 comp rx=0 tx=0).stop 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 -- 172.21.15.99:0/3827417563 shutdown_connections 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3827417563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c10f2a0 0x7f119c1116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3827417563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f119c06b800 0x7f119c06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3827417563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c103940 0x7f119c06b2c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 -- 172.21.15.99:0/3827417563 >> 172.21.15.99:0/3827417563 conn(0x7f119c0786a0 msgr2=0x7f119c078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.685+0000 7f11a37c8640 1 -- 172.21.15.99:0/3827417563 shutdown_connections 2024-05-14T17:16:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 -- 172.21.15.99:0/3827417563 wait complete. 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 Processor -- start 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 -- start start 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 0x7f119c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c103940 0x7f119c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f119c10f2a0 0x7f119c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f119c114ec0 con 0x7f119c06b800 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f119c114d40 con 0x7f119c103940 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a37c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f119c115040 con 0x7f119c10f2a0 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.686+0000 7f11a1564640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 0x7f119c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 0x7f119c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39688/0 (socket says 172.21.15.99:39688) 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 -- 172.21.15.99:0/3385092726 learned_addr learned my addr 172.21.15.99:0/3385092726 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a0d63640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c103940 0x7f119c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f119c10f2a0 0x7f119c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 -- 172.21.15.99:0/3385092726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f119c10f2a0 msgr2=0x7f119c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f119c10f2a0 0x7f119c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 -- 172.21.15.99:0/3385092726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c103940 msgr2=0x7f119c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c103940 0x7f119c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 -- 172.21.15.99:0/3385092726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c1a0370 con 0x7f119c06b800 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a1564640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 0x7f119c19b380 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1198005940 tx=0x7f11980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11927fc640 1 -- 172.21.15.99:0/3385092726 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119803f030 con 0x7f119c06b800 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11927fc640 1 -- 172.21.15.99:0/3385092726 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11980044c0 con 0x7f119c06b800 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c1a0600 con 0x7f119c06b800 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11927fc640 1 -- 172.21.15.99:0/3385092726 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198035070 con 0x7f119c06b800 2024-05-14T17:16:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.687+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c1a7ef0 con 0x7f119c06b800 2024-05-14T17:16:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11927fc640 1 -- 172.21.15.99:0/3385092726 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1198042030 con 0x7f119c06b800 2024-05-14T17:16:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 Processor -- start 2024-05-14T17:16:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 -- start start 2024-05-14T17:16:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d264cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d271ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55673d1d7c30 con 0x55673d2648f0 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55673d1c77d0 con 0x55673d26fad0 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.690+0000 7f11a65c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55673d1cb810 con 0x55673d265210 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d264cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d264cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39692/0 (socket says 172.21.15.99:39692) 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 -- 172.21.15.99:0/3869336928 learned_addr learned my addr 172.21.15.99:0/3869336928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1564640 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 -- 172.21.15.99:0/3869336928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 msgr2=0x55673d26f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a0d63640 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d271ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 -- 172.21.15.99:0/3869336928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 msgr2=0x55673d271ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d271ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 -- 172.21.15.99:0/3869336928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673cedc210 con 0x55673d2648f0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1564640 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11927fc640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f11880b2a10 0x7f11880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d264cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1194009f50 tx=0x7f11940118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbe80d4750057dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11a1564640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f11880b2a10 0x7f11880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11927fc640 1 -- 172.21.15.99:0/3385092726 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f1198102580 con 0x7f119c06b800 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11877fe640 1 -- 172.21.15.99:0/3869336928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194004d20 con 0x55673d2648f0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11877fe640 1 -- 172.21.15.99:0/3869336928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194004050 con 0x55673d2648f0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.691+0000 7f11877fe640 1 -- 172.21.15.99:0/3869336928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194005220 con 0x55673d2648f0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 -- 172.21.15.99:0/3869336928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 msgr2=0x55673d264cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a1564640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f11880b2a10 0x7f11880b4ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f11980ffd80 tx=0x7f11980d2040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d264cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1194009f50 tx=0x7f11940118d0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 -- 172.21.15.99:0/3869336928 shutdown_connections 2024-05-14T17:16:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d271ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3869336928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d264cd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 -- 172.21.15.99:0/3869336928 >> 172.21.15.99:0/3869336928 conn(0x55673d260e00 msgr2=0x55673d2611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 -- 172.21.15.99:0/3869336928 shutdown_connections 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 -- 172.21.15.99:0/3869336928 wait complete. 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.692+0000 7f11a65c2740 1 -- start start 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 Processor -- start 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d26f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d26cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55673cead790 con 0x55673d2648f0 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55673d183890 con 0x55673d26fad0 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55673d2727a0 con 0x55673d265210 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a65c2740 10 client.0 _ll_register_callbacks cb 0x55673d186d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d26f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d26f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39700/0 (socket says 172.21.15.99:39700) 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 -- 172.21.15.99:0/1254357177 learned_addr learned my addr 172.21.15.99:0/1254357177 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a0d63640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d26cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 -- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 msgr2=0x55673d26c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 -- 172.21.15.99:0/1254357177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 msgr2=0x55673d26cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d26cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.693+0000 7f11a1d65640 1 -- 172.21.15.99:0/1254357177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673cfa8620 con 0x55673d2648f0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a0d63640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a1d65640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d26f3c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1194005c40 tx=0x7f1194005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fd0ec083b51b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f1184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f1184ff9640 1 -- 172.21.15.99:0/1254357177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119402b020 con 0x55673d2648f0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f1184ff9640 1 -- 172.21.15.99:0/1254357177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f1194005040 con 0x55673d2648f0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194004050 con 0x55673d2648f0 2024-05-14T17:16:33.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f1184ff9640 1 -- 172.21.15.99:0/1254357177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1194005240 con 0x55673d2648f0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 10 client.16884 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55673d27e160 con 0x55673d2648f0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f1184ff9640 1 -- 172.21.15.99:0/1254357177 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194004050 con 0x55673d2648f0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f1184ff9640 1 -- 172.21.15.99:0/1254357177 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f11940279a0 con 0x55673d2648f0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 10 client.16884 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 20 client.16884 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d27f5c0 0x55673d2819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 4 client.16884 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.694+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55673d1d9b80 con 0x55673d27f5c0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.695+0000 7f11a0d63640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d27f5c0 0x55673d2819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.695+0000 7f1184ff9640 10 client.16884 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.695+0000 7f11a0d63640 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d27f5c0 0x55673d2819b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da0e96967f6e23a server_cookie=b4e0a8803f2d3d5f in_seq=0 out_seq=0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f1184ff9640 1 -- 172.21.15.99:0/1254357177 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55673d1d9b80 con 0x55673d27f5c0 2024-05-14T17:16:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f1184ff9640 10 client.16884 handle_command_reply: tid=0 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 client.16884 shutdown 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 msgr2=0x55673d26f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d26f3c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1194005c40 tx=0x7f1194005bd0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 shutdown_connections 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d27f5c0 0x55673d2819b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d26fad0 0x55673d26cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d265210 0x55673d26c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f1184ff9640 0 client.16884 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 --2- 172.21.15.99:0/1254357177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d2648f0 0x55673d26f3c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 >> 172.21.15.99:0/1254357177 conn(0x55673d260e00 msgr2=0x55673d260100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 shutdown_connections 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 1 -- 172.21.15.99:0/1254357177 wait complete. 2024-05-14T17:16:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.696+0000 7f11a65c2740 20 client.16884 trim_cache size 0 max 16384 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 Processor -- start 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 -- start start 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d27b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d267b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 0x55673d26eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1b3140 con 0x55673d265210 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1b0a10 con 0x55673d2648f0 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a65c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1ae880 con 0x55673d26fad0 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d267b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d267b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39708/0 (socket says 172.21.15.99:39708) 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 -- 172.21.15.99:0/2831852552 learned_addr learned my addr 172.21.15.99:0/2831852552 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:33.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 -- 172.21.15.99:0/2831852552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 msgr2=0x55673d26eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 0x55673d26eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 -- 172.21.15.99:0/2831852552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 msgr2=0x55673d27b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d27b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 -- 172.21.15.99:0/2831852552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673d260c30 con 0x55673d265210 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f11a0d63640 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d267b20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55673d261f90 tx=0x7f118c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338ed32dfd5a4d7b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f1186ffd640 1 -- 172.21.15.99:0/2831852552 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c004030 con 0x55673d265210 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.712+0000 7f1186ffd640 1 -- 172.21.15.99:0/2831852552 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118c005020 con 0x55673d265210 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f1186ffd640 1 -- 172.21.15.99:0/2831852552 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c004cd0 con 0x55673d265210 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 -- 172.21.15.99:0/2831852552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 msgr2=0x55673d267b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d267b20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55673d261f90 tx=0x7f118c0128e0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 -- 172.21.15.99:0/2831852552 shutdown_connections 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 0x55673d26eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d267b20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 --2- 172.21.15.99:0/2831852552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d27b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 -- 172.21.15.99:0/2831852552 >> 172.21.15.99:0/2831852552 conn(0x55673d260e00 msgr2=0x55673d267410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 -- 172.21.15.99:0/2831852552 shutdown_connections 2024-05-14T17:16:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 -- 172.21.15.99:0/2831852552 wait complete. 2024-05-14T17:16:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 -- start start 2024-05-14T17:16:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.713+0000 7f11a65c2740 1 Processor -- start 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d2e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d2e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 0x55673d2774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1aeed0 con 0x55673d265210 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1b0340 con 0x55673d2648f0 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1af5a0 con 0x55673d26fad0 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a65c2740 10 client.0 _ll_register_callbacks cb 0x7f11940042f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d2e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d2e2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39718/0 (socket says 172.21.15.99:39718) 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 -- 172.21.15.99:0/3545771554 learned_addr learned my addr 172.21.15.99:0/3545771554 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d2e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 -- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 msgr2=0x55673d2774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 0x55673d2774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 -- 172.21.15.99:0/3545771554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 msgr2=0x55673d2e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d2e1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 -- 172.21.15.99:0/3545771554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673d277b60 con 0x55673d265210 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d2e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f11a0d63640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d2e2340 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f118c0051d0 tx=0x7f118c005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf03bbc4c05269 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.714+0000 7f1184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f1184ff9640 1 -- 172.21.15.99:0/3545771554 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c004050 con 0x55673d265210 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f1184ff9640 1 -- 172.21.15.99:0/3545771554 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f118c0041f0 con 0x55673d265210 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118c005020 con 0x55673d265210 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f1184ff9640 1 -- 172.21.15.99:0/3545771554 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f118c033050 con 0x55673d265210 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 10 client.16890 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55673d2e96e0 con 0x55673d265210 2024-05-14T17:16:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f1184ff9640 1 -- 172.21.15.99:0/3545771554 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118c005020 con 0x55673d265210 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f1184ff9640 1 -- 172.21.15.99:0/3545771554 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7f118c02e030 con 0x55673d265210 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 10 client.16890 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 20 client.16890 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d2e9e80 0x55673d2ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 4 client.16890 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55673d1d3360 con 0x55673d2e9e80 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.715+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d2e9e80 0x55673d2ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.716+0000 7f1184ff9640 10 client.16890 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.716+0000 7f11a1d65640 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d2e9e80 0x55673d2ec2a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6688b34897439de0 server_cookie=f30b6a1f189e5566 in_seq=0 out_seq=0 2024-05-14T17:16:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.766+0000 7f1184ff9640 1 -- 172.21.15.99:0/3545771554 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55673d1d3360 con 0x55673d2e9e80 2024-05-14T17:16:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.766+0000 7f1184ff9640 10 client.16890 handle_command_reply: tid=0 2024-05-14T17:16:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.766+0000 7f11a65c2740 1 client.16890 shutdown 2024-05-14T17:16:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.766+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 msgr2=0x55673d2e2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d2e2340 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f118c0051d0 tx=0x7f118c005200 comp rx=0 tx=0).stop 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 shutdown_connections 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55673d2e9e80 0x55673d2ec2a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673d26fad0 0x55673d2774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55673d265210 0x55673d2e2340 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f1184ff9640 0 client.16890 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 --2- 172.21.15.99:0/3545771554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55673d2648f0 0x55673d2e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 >> 172.21.15.99:0/3545771554 conn(0x55673d260e00 msgr2=0x55673d267ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 shutdown_connections 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 1 -- 172.21.15.99:0/3545771554 wait complete. 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a65c2740 20 client.16890 trim_cache size 0 max 16384 2024-05-14T17:16:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:16:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:16:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:16:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f11880b2a10 msgr2=0x7f11880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f11880b2a10 0x7f11880b4ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f11980ffd80 tx=0x7f11980d2040 comp rx=0 tx=0).stop 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 msgr2=0x7f119c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.767+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 0x7f119c19b380 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1198005940 tx=0x7f11980040c0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 shutdown_connections 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f11880b2a10 0x7f11880b4ed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f119c10f2a0 0x7f119c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c103940 0x7f119c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 --2- 172.21.15.99:0/3385092726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c06b800 0x7f119c19b380 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 >> 172.21.15.99:0/3385092726 conn(0x7f119c0786a0 msgr2=0x7f119c1100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 shutdown_connections 2024-05-14T17:16:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:33.768+0000 7f11a37c8640 1 -- 172.21.15.99:0/3385092726 wait complete. 2024-05-14T17:16:34.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 502633 3.76 MB/sec execute 526 sec latency 3138.043 ms 2024-05-14T17:16:35.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:35 smithi150 ceph-mon[36204]: pgmap v407: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 967 KiB/s wr, 114 op/s 2024-05-14T17:16:35.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:35 smithi099 ceph-mon[31851]: pgmap v407: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 967 KiB/s wr, 114 op/s 2024-05-14T17:16:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:35 smithi193 ceph-mon[35962]: pgmap v407: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 967 KiB/s wr, 114 op/s 2024-05-14T17:16:35.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 504887 3.78 MB/sec execute 527 sec latency 3189.162 ms 2024-05-14T17:16:36.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 506183 3.78 MB/sec execute 528 sec latency 196.568 ms 2024-05-14T17:16:37.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:37 smithi150 ceph-mon[36204]: pgmap v408: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 638 KiB/s wr, 73 op/s 2024-05-14T17:16:37.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:37 smithi099 ceph-mon[31851]: pgmap v408: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 638 KiB/s wr, 73 op/s 2024-05-14T17:16:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:37 smithi193 ceph-mon[35962]: pgmap v408: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 638 KiB/s wr, 73 op/s 2024-05-14T17:16:37.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 506183 3.77 MB/sec execute 529 sec latency 1196.698 ms 2024-05-14T17:16:38.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 506183 3.77 MB/sec execute 530 sec latency 2196.781 ms 2024-05-14T17:16:39.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:39 smithi150 ceph-mon[36204]: pgmap v409: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 638 KiB/s wr, 73 op/s 2024-05-14T17:16:39.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:39 smithi099 ceph-mon[31851]: pgmap v409: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 638 KiB/s wr, 73 op/s 2024-05-14T17:16:39.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:39 smithi193 ceph-mon[35962]: pgmap v409: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 638 KiB/s wr, 73 op/s 2024-05-14T17:16:39.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 506183 3.76 MB/sec execute 531 sec latency 3196.834 ms 2024-05-14T17:16:40.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 508423 3.78 MB/sec execute 532 sec latency 3239.331 ms 2024-05-14T17:16:41.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:41 smithi150 ceph-mon[36204]: pgmap v410: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-05-14T17:16:41.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:41 smithi099 ceph-mon[31851]: pgmap v410: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-05-14T17:16:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:41 smithi193 ceph-mon[35962]: pgmap v410: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-05-14T17:16:41.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 509726 3.78 MB/sec execute 533 sec latency 116.570 ms 2024-05-14T17:16:42.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 509726 3.77 MB/sec execute 534 sec latency 1116.651 ms 2024-05-14T17:16:43.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:43 smithi150 ceph-mon[36204]: pgmap v411: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 743 KiB/s wr, 101 op/s 2024-05-14T17:16:43.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:43 smithi099 ceph-mon[31851]: pgmap v411: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 743 KiB/s wr, 101 op/s 2024-05-14T17:16:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:43 smithi193 ceph-mon[35962]: pgmap v411: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 743 KiB/s wr, 101 op/s 2024-05-14T17:16:43.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 509726 3.77 MB/sec execute 535 sec latency 2116.708 ms 2024-05-14T17:16:44.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 509726 3.76 MB/sec execute 536 sec latency 3116.759 ms 2024-05-14T17:16:45.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:45 smithi150 ceph-mon[36204]: pgmap v412: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-05-14T17:16:45.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:45 smithi099 ceph-mon[31851]: pgmap v412: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-05-14T17:16:45.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:45 smithi193 ceph-mon[35962]: pgmap v412: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-05-14T17:16:45.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 512017 3.78 MB/sec execute 537 sec latency 3123.893 ms 2024-05-14T17:16:46.563 INFO:tasks.workunit.client.0.smithi099.stdout: 1 513272 3.78 MB/sec execute 538 sec latency 310.482 ms 2024-05-14T17:16:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 -- 172.21.15.99:0/4014881551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa754102370 msgr2=0x7fa75410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 --2- 172.21.15.99:0/4014881551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa754102370 0x7fa75410e830 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009cd0 tx=0x7fa74c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 -- 172.21.15.99:0/4014881551 shutdown_connections 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 --2- 172.21.15.99:0/4014881551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa754102370 0x7fa75410e830 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 --2- 172.21.15.99:0/4014881551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7541019b0 0x7fa754101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 --2- 172.21.15.99:0/4014881551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa754107970 0x7fa754107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 -- 172.21.15.99:0/4014881551 >> 172.21.15.99:0/4014881551 conn(0x7fa754078b60 msgr2=0x7fa754078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 -- 172.21.15.99:0/4014881551 shutdown_connections 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.858+0000 7fa75bbda640 1 -- 172.21.15.99:0/4014881551 wait complete. 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 Processor -- start 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 -- start start 2024-05-14T17:16:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 0x7fa754199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa754102370 0x7fa754199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa754107970 0x7fa75419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa754111c30 con 0x7fa7541019b0 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa754111ab0 con 0x7fa754107970 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75bbda640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa754111db0 con 0x7fa754102370 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa759976640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 0x7fa754199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa759976640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 0x7fa754199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57598/0 (socket says 172.21.15.99:57598) 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa759976640 1 -- 172.21.15.99:0/1302644653 learned_addr learned my addr 172.21.15.99:0/1302644653 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa759175640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa754102370 0x7fa754199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.859+0000 7fa75a177640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa754107970 0x7fa75419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa759976640 1 -- 172.21.15.99:0/1302644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa754102370 msgr2=0x7fa754199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa759976640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa754102370 0x7fa754199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa759976640 1 -- 172.21.15.99:0/1302644653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa754107970 msgr2=0x7fa75419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa759976640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa754107970 0x7fa75419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa759976640 1 -- 172.21.15.99:0/1302644653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75419e190 con 0x7fa7541019b0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa75a177640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa754107970 0x7fa75419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa759976640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 0x7fa754199240 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9f0 tx=0x7fa75000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa74affd640 1 -- 172.21.15.99:0/1302644653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75000cd80 con 0x7fa7541019b0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa74affd640 1 -- 172.21.15.99:0/1302644653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7500054d0 con 0x7fa7541019b0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75419e480 con 0x7fa7541019b0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa74affd640 1 -- 172.21.15.99:0/1302644653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750010690 con 0x7fa7541019b0 2024-05-14T17:16:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.860+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7541a5d10 con 0x7fa7541019b0 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.862+0000 7fa74affd640 1 -- 172.21.15.99:0/1302644653 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa750004e90 con 0x7fa7541019b0 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 Processor -- start 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 -- start start 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917eb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ec19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ec42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a917e25c30 con 0x55a917eb30f0 2024-05-14T17:16:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a917e157d0 con 0x55a917eb27d0 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a917e19810 con 0x55a917ec1ee0 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ec19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ec19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57600/0 (socket says 172.21.15.99:57600) 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 -- 172.21.15.99:0/3879524782 learned_addr learned my addr 172.21.15.99:0/3879524782 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa75a177640 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917eb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759175640 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ec42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 -- 172.21.15.99:0/3879524782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 msgr2=0x55a917ec42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ec42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 -- 172.21.15.99:0/3879524782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 msgr2=0x55a917eb2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917eb2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa74affd640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7400b2a10 0x7fa7400b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759976640 1 -- 172.21.15.99:0/3879524782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a917b2a210 con 0x55a917eb30f0 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.863+0000 7fa759175640 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ec42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.864+0000 7fa759175640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7400b2a10 0x7fa7400b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.864+0000 7fa759976640 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ec19a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa7500e0190 tx=0x7fa7500a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a223f8bf46b4e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.864+0000 7fa74affd640 1 -- 172.21.15.99:0/1302644653 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fa7500e2b00 con 0x7fa7541019b0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.864+0000 7fa73ffff640 1 -- 172.21.15.99:0/3879524782 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500a9bb0 con 0x55a917eb30f0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.864+0000 7fa73ffff640 1 -- 172.21.15.99:0/3879524782 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7500a98f0 con 0x55a917eb30f0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.864+0000 7fa73ffff640 1 -- 172.21.15.99:0/3879524782 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750005180 con 0x55a917eb30f0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3879524782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 msgr2=0x55a917ec19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa759175640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7400b2a10 0x7fa7400b4ed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55a917eb3d60 tx=0x7fa74400b650 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ec19a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa7500e0190 tx=0x7fa7500a64f0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3879524782 shutdown_connections 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ec42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ec19a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3879524782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917eb2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3879524782 >> 172.21.15.99:0/3879524782 conn(0x55a917eaee00 msgr2=0x55a917eaf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3879524782 shutdown_connections 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3879524782 wait complete. 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 -- start start 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.865+0000 7fa75e9d0740 1 Processor -- start 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917ec1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ebeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ebf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a917afb790 con 0x55a917eb30f0 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a917dd1890 con 0x55a917eb27d0 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a917ec4bb0 con 0x55a917ec1ee0 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 10 client.0 _ll_register_callbacks cb 0x55a917dd4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ebeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ebeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57612/0 (socket says 172.21.15.99:57612) 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 -- 172.21.15.99:0/1288297084 learned_addr learned my addr 172.21.15.99:0/1288297084 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75a177640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ebf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759976640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917ec1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 -- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 msgr2=0x55a917ebf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ebf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 -- 172.21.15.99:0/1288297084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 msgr2=0x55a917ec1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917ec1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 -- 172.21.15.99:0/1288297084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a917bf6620 con 0x55a917eb30f0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa759175640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ebeb60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa7440126d0 tx=0x7fa744004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828f051eecf867c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa73d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa73d7fa640 1 -- 172.21.15.99:0/1288297084 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744023070 con 0x55a917eb30f0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa73d7fa640 1 -- 172.21.15.99:0/1288297084 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fa744011ae0 con 0x55a917eb30f0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7500a98f0 con 0x55a917eb30f0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.866+0000 7fa73d7fa640 1 -- 172.21.15.99:0/1288297084 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa74402f050 con 0x55a917eb30f0 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 10 client.16902 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a917ed0640 con 0x55a917eb30f0 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa73d7fa640 1 -- 172.21.15.99:0/1288297084 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7500a98f0 con 0x55a917eb30f0 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa73d7fa640 1 -- 172.21.15.99:0/1288297084 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fa74400f040 con 0x55a917eb30f0 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 10 client.16902 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 20 client.16902 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917ed1b70 0x55a917ed3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 4 client.16902 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a917e27b80 con 0x55a917ed1b70 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.867+0000 7fa759976640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917ed1b70 0x55a917ed3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa73d7fa640 10 client.16902 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa759976640 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917ed1b70 0x55a917ed3f60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c6a2a726a7d6d4 server_cookie=37ba5110f357de60 in_seq=0 out_seq=0 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa73d7fa640 1 -- 172.21.15.99:0/1288297084 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a917e27b80 con 0x55a917ed1b70 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa73d7fa640 10 client.16902 handle_command_reply: tid=0 2024-05-14T17:16:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa75e9d0740 1 client.16902 shutdown 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 msgr2=0x55a917ebeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.868+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ebeb60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa7440126d0 tx=0x7fa744004730 comp rx=0 tx=0).stop 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 shutdown_connections 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917ed1b70 0x55a917ed3f60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917ec1ee0 0x55a917ebf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917ebeb60 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/1288297084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb27d0 0x55a917ec1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa73d7fa640 0 client.16902 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 >> 172.21.15.99:0/1288297084 conn(0x55a917eaee00 msgr2=0x55a917eb4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 shutdown_connections 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 1 -- 172.21.15.99:0/1288297084 wait complete. 2024-05-14T17:16:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.869+0000 7fa75e9d0740 20 client.16902 trim_cache size 0 max 16384 2024-05-14T17:16:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 Processor -- start 2024-05-14T17:16:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 -- start start 2024-05-14T17:16:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ecdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb30f0 0x55a917ec13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 0x55a917ed9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7541aecc0 con 0x55a917ec1ee0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa7541ac590 con 0x55a917eb30f0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7541aa400 con 0x55a917eb27d0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 0x55a917ed9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 0x55a917ed9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57626/0 (socket says 172.21.15.99:57626) 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 -- 172.21.15.99:0/2039045466 learned_addr learned my addr 172.21.15.99:0/2039045466 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa759976640 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ecdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa759175640 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb30f0 0x55a917ec13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 -- 172.21.15.99:0/2039045466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 msgr2=0x55a917ecdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ecdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 -- 172.21.15.99:0/2039045466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb30f0 msgr2=0x55a917ec13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb30f0 0x55a917ec13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 -- 172.21.15.99:0/2039045466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a917eaa830 con 0x55a917ec1ee0 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa759175640 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb30f0 0x55a917ec13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:16:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.884+0000 7fa75a177640 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 0x55a917ed9810 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009cd0 tx=0x7fa74c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6042f8a51899f9ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa73f7fe640 1 -- 172.21.15.99:0/2039045466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c031620 con 0x55a917ec1ee0 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa73f7fe640 1 -- 172.21.15.99:0/2039045466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa74c036040 con 0x55a917ec1ee0 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa73f7fe640 1 -- 172.21.15.99:0/2039045466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0043a0 con 0x55a917ec1ee0 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 -- 172.21.15.99:0/2039045466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 msgr2=0x55a917ed9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 0x55a917ed9810 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009cd0 tx=0x7fa74c009a20 comp rx=0 tx=0).stop 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 -- 172.21.15.99:0/2039045466 shutdown_connections 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917ec1ee0 0x55a917ed9810 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917eb30f0 0x55a917ec13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/2039045466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ecdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 -- 172.21.15.99:0/2039045466 >> 172.21.15.99:0/2039045466 conn(0x55a917eaee00 msgr2=0x55a917eae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 -- 172.21.15.99:0/2039045466 shutdown_connections 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 -- 172.21.15.99:0/2039045466 wait complete. 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.885+0000 7fa75e9d0740 1 -- start start 2024-05-14T17:16:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 Processor -- start 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ec9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917eca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917ec1ee0 0x55a917eb60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7541aaa50 con 0x55a917eb30f0 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa7541abec0 con 0x55a917ec1ee0 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7541ab120 con 0x55a917eb27d0 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75e9d0740 10 client.0 _ll_register_callbacks cb 0x7fa74400bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa759175640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917eca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa759976640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ec9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa759175640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917eca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57632/0 (socket says 172.21.15.99:57632) 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa759175640 1 -- 172.21.15.99:0/3712520685 learned_addr learned my addr 172.21.15.99:0/3712520685 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:16:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.886+0000 7fa75a177640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917ec1ee0 0x55a917eb60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759976640 1 -- 172.21.15.99:0/3712520685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917ec1ee0 msgr2=0x55a917eb60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759976640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917ec1ee0 0x55a917eb60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759976640 1 -- 172.21.15.99:0/3712520685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 msgr2=0x55a917eca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759976640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917eca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759976640 1 -- 172.21.15.99:0/3712520685 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a917eb6890 con 0x55a917eb27d0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759175640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917eca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa759976640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ec9eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7500a5e60 tx=0x7fa750004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99877bf035984e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa73d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa73d7fa640 1 -- 172.21.15.99:0/3712520685 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500ad410 con 0x55a917eb27d0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa73d7fa640 1 -- 172.21.15.99:0/3712520685 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fa7500e04d0 con 0x55a917eb27d0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c036040 con 0x55a917eb27d0 2024-05-14T17:16:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa73d7fa640 1 -- 172.21.15.99:0/3712520685 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa7500a9550 con 0x55a917eb27d0 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa75e9d0740 10 client.26218 fetch_fsmap learned FSMap version 42 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a917ec7ad0 con 0x55a917eb27d0 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.887+0000 7fa73d7fa640 1 -- 172.21.15.99:0/3712520685 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa74c036040 con 0x55a917eb27d0 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa73d7fa640 1 -- 172.21.15.99:0/3712520685 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fa7500a5750 con 0x55a917eb27d0 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa75e9d0740 10 client.26218 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa75e9d0740 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa75e9d0740 20 client.26218 populate_metadata read hostname 'smithi099' 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917f40bb0 0x55a917f42fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa75e9d0740 4 client.26218 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a917e21360 con 0x55a917f40bb0 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa759175640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917f40bb0 0x55a917f42fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa73d7fa640 10 client.26218 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:46.888+0000 7fa759175640 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917f40bb0 0x55a917f42fa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7d3d87dcb22a47 server_cookie=496084ae19ccb629 in_seq=0 out_seq=0 2024-05-14T17:16:47.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:47 smithi150 ceph-mon[36204]: pgmap v413: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-14T17:16:47.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:47 smithi099 ceph-mon[31851]: pgmap v413: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-14T17:16:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:47 smithi193 ceph-mon[35962]: pgmap v413: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-14T17:16:47.562 INFO:tasks.workunit.client.0.smithi099.stdout: 1 513272 3.77 MB/sec execute 539 sec latency 1310.539 ms 2024-05-14T17:16:48.563 INFO:tasks.workunit.client.0.smithi099.stdout: 1 513272 3.77 MB/sec execute 540 sec latency 2310.655 ms 2024-05-14T17:16:49.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:49 smithi150 ceph-mon[36204]: pgmap v414: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-14T17:16:49.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:49 smithi099 ceph-mon[31851]: pgmap v414: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-14T17:16:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:49 smithi193 ceph-mon[35962]: pgmap v414: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-14T17:16:49.563 INFO:tasks.workunit.client.0.smithi099.stdout: 1 513272 3.76 MB/sec execute 541 sec latency 3310.692 ms 2024-05-14T17:16:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa73d7fa640 1 -- 172.21.15.99:0/3712520685 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a917e21360 con 0x55a917f40bb0 2024-05-14T17:16:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa73d7fa640 10 client.26218 handle_command_reply: tid=0 2024-05-14T17:16:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 client.26218 shutdown 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 msgr2=0x55a917ec9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ec9eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7500a5e60 tx=0x7fa750004a90 comp rx=0 tx=0).stop 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 shutdown_connections 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55a917f40bb0 0x55a917f42fa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a917ec1ee0 0x55a917eb60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a917eb30f0 0x55a917eca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa73d7fa640 0 client.26218 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 --2- 172.21.15.99:0/3712520685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a917eb27d0 0x55a917ec9eb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 >> 172.21.15.99:0/3712520685 conn(0x55a917eaee00 msgr2=0x55a917ebe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 shutdown_connections 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.616+0000 7fa75e9d0740 1 -- 172.21.15.99:0/3712520685 wait complete. 2024-05-14T17:16:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75e9d0740 20 client.26218 trim_cache size 0 max 16384 2024-05-14T17:16:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:16:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:16:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7400b2a10 msgr2=0x7fa7400b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7400b2a10 0x7fa7400b4ed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55a917eb3d60 tx=0x7fa74400b650 comp rx=0 tx=0).stop 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 msgr2=0x7fa754199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 0x7fa754199240 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9f0 tx=0x7fa75000edf0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 shutdown_connections 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa7400b2a10 0x7fa7400b4ed0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa754107970 0x7fa75419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa754102370 0x7fa754199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 --2- 172.21.15.99:0/1302644653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7541019b0 0x7fa754199240 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.617+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 >> 172.21.15.99:0/1302644653 conn(0x7fa754078b60 msgr2=0x7fa75406b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.618+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 shutdown_connections 2024-05-14T17:16:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:16:49.618+0000 7fa75bbda640 1 -- 172.21.15.99:0/1302644653 wait complete. 2024-05-14T17:16:50.563 INFO:tasks.workunit.client.0.smithi099.stdout: 1 515516 3.78 MB/sec execute 542 sec latency 3358.732 ms 2024-05-14T17:16:51.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:51 smithi150 ceph-mon[36204]: pgmap v415: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-14T17:16:51.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:51 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:16:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:51 smithi099 ceph-mon[31851]: pgmap v415: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-14T17:16:51.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:51 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:16:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:51 smithi193 ceph-mon[35962]: pgmap v415: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-14T17:16:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:51 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:16:51.563 INFO:tasks.workunit.client.0.smithi099.stdout: 1 516816 3.78 MB/sec execute 543 sec latency 113.501 ms 2024-05-14T17:16:52.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 516816 3.77 MB/sec execute 544 sec latency 1113.539 ms 2024-05-14T17:16:53.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:53 smithi150 ceph-mon[36204]: pgmap v416: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 630 KiB/s wr, 74 op/s 2024-05-14T17:16:53.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:53 smithi099 ceph-mon[31851]: pgmap v416: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 630 KiB/s wr, 74 op/s 2024-05-14T17:16:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:53 smithi193 ceph-mon[35962]: pgmap v416: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 630 KiB/s wr, 74 op/s 2024-05-14T17:16:53.563 INFO:tasks.workunit.client.0.smithi099.stdout: 1 516816 3.77 MB/sec execute 545 sec latency 2114.160 ms 2024-05-14T17:16:54.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 516816 3.76 MB/sec execute 546 sec latency 3114.206 ms 2024-05-14T17:16:55.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:55 smithi150 ceph-mon[36204]: pgmap v417: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-14T17:16:55.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:55 smithi099 ceph-mon[31851]: pgmap v417: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-14T17:16:55.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:55 smithi193 ceph-mon[35962]: pgmap v417: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-05-14T17:16:55.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 518986 3.78 MB/sec execute 547 sec latency 3154.373 ms 2024-05-14T17:16:56.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 520365 3.78 MB/sec execute 548 sec latency 39.933 ms 2024-05-14T17:16:57.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:57 smithi150 ceph-mon[36204]: pgmap v418: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 947 KiB/s wr, 92 op/s 2024-05-14T17:16:57.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:57 smithi099 ceph-mon[31851]: pgmap v418: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 947 KiB/s wr, 92 op/s 2024-05-14T17:16:57.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:57 smithi193 ceph-mon[35962]: pgmap v418: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 947 KiB/s wr, 92 op/s 2024-05-14T17:16:57.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 520365 3.77 MB/sec execute 549 sec latency 1040.024 ms 2024-05-14T17:16:58.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 520365 3.77 MB/sec execute 550 sec latency 2040.068 ms 2024-05-14T17:16:59.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:16:59 smithi099 ceph-mon[31851]: pgmap v419: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 948 KiB/s wr, 92 op/s 2024-05-14T17:16:59.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:16:59 smithi193 ceph-mon[35962]: pgmap v419: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 948 KiB/s wr, 92 op/s 2024-05-14T17:16:59.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 520365 3.76 MB/sec execute 551 sec latency 3040.107 ms 2024-05-14T17:16:59.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:16:59 smithi150 ceph-mon[36204]: pgmap v419: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 948 KiB/s wr, 92 op/s 2024-05-14T17:17:00.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 522034 3.77 MB/sec execute 552 sec latency 3051.705 ms 2024-05-14T17:17:01.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:01 smithi099 ceph-mon[31851]: pgmap v420: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-05-14T17:17:01.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:01 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:17:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:01 smithi193 ceph-mon[35962]: pgmap v420: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-05-14T17:17:01.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:01 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:17:01.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 523598 3.78 MB/sec execute 553 sec latency 26.564 ms 2024-05-14T17:17:01.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:01 smithi150 ceph-mon[36204]: pgmap v420: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-05-14T17:17:01.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:01 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:17:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:02 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:17:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:02 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:17:02.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:02 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:17:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:02 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:17:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:02 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:17:02.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:02 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:17:02.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 523906 3.77 MB/sec execute 554 sec latency 667.162 ms 2024-05-14T17:17:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:02 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:17:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:02 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:17:02.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:02 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 -- 172.21.15.99:0/4174447056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda8107e30 msgr2=0x7fcda810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/4174447056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda8107e30 0x7fcda810f9b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009900 tx=0x7fcda002a410 comp rx=0 tx=0).stop 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 -- 172.21.15.99:0/4174447056 shutdown_connections 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/4174447056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda8107e30 0x7fcda810f9b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/4174447056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcda8101b70 0x7fcda8107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/4174447056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda81011c0 0x7fcda81015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 -- 172.21.15.99:0/4174447056 >> 172.21.15.99:0/4174447056 conn(0x7fcda8077eb0 msgr2=0x7fcda80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 -- 172.21.15.99:0/4174447056 shutdown_connections 2024-05-14T17:17:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.709+0000 7fcdac9f7640 1 -- 172.21.15.99:0/4174447056 wait complete. 2024-05-14T17:17:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 Processor -- start 2024-05-14T17:17:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 -- start start 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 0x7fcda8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda8107e30 0x7fcda81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcda819da20 0x7fcda819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112d40 con 0x7fcda81011c0 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112bc0 con 0x7fcda8107e30 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcdac9f7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8112ec0 con 0x7fcda819da20 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 0x7fcda8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 0x7fcda8199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33954/0 (socket says 172.21.15.99:33954) 2024-05-14T17:17:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 -- 172.21.15.99:0/3581410121 learned_addr learned my addr 172.21.15.99:0/3581410121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda8107e30 0x7fcda81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcda819da20 0x7fcda819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 -- 172.21.15.99:0/3581410121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcda819da20 msgr2=0x7fcda819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcda819da20 0x7fcda819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 -- 172.21.15.99:0/3581410121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda8107e30 msgr2=0x7fcda81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda8107e30 0x7fcda81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 -- 172.21.15.99:0/3581410121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda819e510 con 0x7fcda81011c0 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.710+0000 7fcda659c640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 0x7fcda8199160 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800ebc0 tx=0x7fcd9800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.711+0000 7fcd977fe640 1 -- 172.21.15.99:0/3581410121 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98010630 con 0x7fcda81011c0 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.711+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda819e800 con 0x7fcda81011c0 2024-05-14T17:17:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.711+0000 7fcd977fe640 1 -- 172.21.15.99:0/3581410121 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd98010c80 con 0x7fcda81011c0 2024-05-14T17:17:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.711+0000 7fcd977fe640 1 -- 172.21.15.99:0/3581410121 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98018b60 con 0x7fcda81011c0 2024-05-14T17:17:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.711+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a5ff0 con 0x7fcda81011c0 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.713+0000 7fcdaf7ec740 1 Processor -- start 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 -- start start 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcd977fe640 1 -- 172.21.15.99:0/3581410121 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcd98018d00 con 0x7fcda81011c0 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 0x55bb2339bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339c210 0x55bb233a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb2330ec30 con 0x55bb2339b8f0 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bb232fe7d0 con 0x55bb2339c210 2024-05-14T17:17:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bb23302810 con 0x55bb233a6ad0 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 0x55bb2339bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 0x55bb2339bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33962/0 (socket says 172.21.15.99:33962) 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 -- 172.21.15.99:0/3808936271 learned_addr learned my addr 172.21.15.99:0/3808936271 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda659c640 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339c210 0x55bb233a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 -- 172.21.15.99:0/3808936271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 msgr2=0x55bb233a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 -- 172.21.15.99:0/3808936271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339c210 msgr2=0x55bb233a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339c210 0x55bb233a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda6d9d640 1 -- 172.21.15.99:0/3808936271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23013210 con 0x55bb2339b8f0 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.714+0000 7fcda659c640 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339c210 0x55bb233a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 0x55bb2339bcd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009900 tx=0x7fcda0032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcfacd6837c78fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:02.722 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcd977fe640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcd900b2a10 0x7fcd900b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcda659c640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcd900b2a10 0x7fcd900b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcd977fe640 1 -- 172.21.15.99:0/3581410121 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fcd980e2cc0 con 0x7fcda81011c0 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcda659c640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcd900b2a10 0x7fcd900b4ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd980e04c0 tx=0x7fcd98000f40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcd94ff9640 1 -- 172.21.15.99:0/3808936271 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda003f070 con 0x55bb2339b8f0 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcd94ff9640 1 -- 172.21.15.99:0/3808936271 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda0004030 con 0x55bb2339b8f0 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.715+0000 7fcd94ff9640 1 -- 172.21.15.99:0/3808936271 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0035070 con 0x55bb2339b8f0 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/3808936271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 msgr2=0x55bb2339bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 0x55bb2339bcd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009900 tx=0x7fcda0032b20 comp rx=0 tx=0).stop 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/3808936271 shutdown_connections 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339c210 0x55bb233a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/3808936271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339b8f0 0x55bb2339bcd0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/3808936271 >> 172.21.15.99:0/3808936271 conn(0x55bb23397e00 msgr2=0x55bb233981e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/3808936271 shutdown_connections 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/3808936271 wait complete. 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.716+0000 7fcdaf7ec740 1 -- start start 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 Processor -- start 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb22fe4790 con 0x55bb2339c210 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bb232ba890 con 0x55bb2339b8f0 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bb233a9740 con 0x55bb233a6ad0 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 10 client.0 _ll_register_callbacks cb 0x55bb232bdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233a3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33964/0 (socket says 172.21.15.99:33964) 2024-05-14T17:17:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4155860961 learned_addr learned my addr 172.21.15.99:0/4155860961 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 msgr2=0x55bb233a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4155860961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 msgr2=0x55bb233a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4155860961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb230df620 con 0x55bb2339c210 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233a3730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcda80751d0 tx=0x7fcd9c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb7ec748b2cb588 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcd8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcd8dffb640 1 -- 172.21.15.99:0/4155860961 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c004020 con 0x55bb2339c210 2024-05-14T17:17:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcd8dffb640 1 -- 172.21.15.99:0/4155860961 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fcd9c0041c0 con 0x55bb2339c210 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.717+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda0004030 con 0x55bb2339c210 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcd8dffb640 1 -- 172.21.15.99:0/4155860961 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcd9c0043c0 con 0x55bb2339c210 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 10 client.16920 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bb233b51b0 con 0x55bb2339c210 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcd8dffb640 1 -- 172.21.15.99:0/4155860961 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda0004030 con 0x55bb2339c210 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcd8dffb640 1 -- 172.21.15.99:0/4155860961 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7fcd9c0039c0 con 0x55bb2339c210 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 10 client.16920 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 20 client.16920 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb233b6730 0x55bb233b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 4 client.16920 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb23310b80 con 0x55bb233b6730 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.718+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb233b6730 0x55bb233b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcd8dffb640 10 client.16920 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb233b6730 0x55bb233b8b20 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd943a85a6ce4c63 server_cookie=dd3867c14f6975d7 in_seq=0 out_seq=0 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcd8dffb640 1 -- 172.21.15.99:0/4155860961 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb23310b80 con 0x55bb233b6730 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcd8dffb640 10 client.16920 handle_command_reply: tid=0 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 client.16920 shutdown 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 msgr2=0x55bb233a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233a3730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcda80751d0 tx=0x7fcd9c00bf90 comp rx=0 tx=0).stop 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 shutdown_connections 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb233b6730 0x55bb233b8b20 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcd8dffb640 0 client.16920 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233a3730 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4155860961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 >> 172.21.15.99:0/4155860961 conn(0x55bb23397e00 msgr2=0x55bb23397100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 shutdown_connections 2024-05-14T17:17:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.719+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4155860961 wait complete. 2024-05-14T17:17:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.720+0000 7fcdaf7ec740 20 client.16920 trim_cache size 0 max 16384 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.734+0000 7fcdaf7ec740 1 Processor -- start 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.734+0000 7fcdaf7ec740 1 -- start start 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.734+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.734+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.734+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81b0f00 con 0x55bb2339c210 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81ae7a0 con 0x55bb2339b8f0 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81ac610 con 0x55bb233a6ad0 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33974/0 (socket says 172.21.15.99:33974) 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4292636918 learned_addr learned my addr 172.21.15.99:0/4292636918 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4292636918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 msgr2=0x55bb233a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4292636918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 msgr2=0x55bb233b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 -- 172.21.15.99:0/4292636918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23393830 con 0x55bb2339c210 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233aed70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c003e30 tx=0x7fcd9c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db3d6b1bcda9c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcd8ffff640 1 -- 172.21.15.99:0/4292636918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c003fa0 con 0x55bb2339c210 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcd8ffff640 1 -- 172.21.15.99:0/4292636918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd9c003aa0 con 0x55bb2339c210 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.735+0000 7fcd8ffff640 1 -- 172.21.15.99:0/4292636918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c010960 con 0x55bb2339c210 2024-05-14T17:17:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4292636918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 msgr2=0x55bb233aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233aed70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c003e30 tx=0x7fcd9c005900 comp rx=0 tx=0).stop 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4292636918 shutdown_connections 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb233a6ad0 0x55bb233a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb2339c210 0x55bb233aed70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/4292636918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb233b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4292636918 >> 172.21.15.99:0/4292636918 conn(0x55bb23397e00 msgr2=0x55bb2339edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4292636918 shutdown_connections 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/4292636918 wait complete. 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- start start 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 Processor -- start 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb2341da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb2339c210 0x55bb2341df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 0x55bb233bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81acc60 con 0x55bb233a6ad0 2024-05-14T17:17:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81ae0d0 con 0x55bb2339b8f0 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81ad330 con 0x55bb2339c210 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcdaf7ec740 10 client.0 _ll_register_callbacks cb 0x7fcd9c00b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcda6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 0x55bb233bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcda6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 0x55bb233bdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33984/0 (socket says 172.21.15.99:33984) 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcda6d9d640 1 -- 172.21.15.99:0/56118553 learned_addr learned my addr 172.21.15.99:0/56118553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.736+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb2339c210 0x55bb2341df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb2341da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 -- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb2339c210 msgr2=0x55bb2341df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb2339c210 0x55bb2341df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 -- 172.21.15.99:0/56118553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 msgr2=0x55bb2341da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb2341da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 -- 172.21.15.99:0/56118553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb233be610 con 0x55bb233a6ad0 2024-05-14T17:17:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcda6d9d640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 0x55bb233bdf20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcda0005c40 tx=0x7fcda0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4945c03c5f89d8bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcd8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcd8dffb640 1 -- 172.21.15.99:0/56118553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda003f040 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd9c003aa0 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcd8dffb640 1 -- 172.21.15.99:0/56118553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fcda0052c70 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcd8dffb640 1 -- 172.21.15.99:0/56118553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcda005f050 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 10 client.16926 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bb23425220 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcd8dffb640 1 -- 172.21.15.99:0/56118553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd9c003aa0 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcd8dffb640 1 -- 172.21.15.99:0/56118553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7fcda00523e0 con 0x55bb233a6ad0 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 10 client.16926 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 20 client.16926 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.737+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb23425580 0x55bb23427970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.738+0000 7fcdaf7ec740 4 client.16926 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:17:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.738+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb2330a360 con 0x55bb23425580 2024-05-14T17:17:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.738+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb23425580 0x55bb23427970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.738+0000 7fcd8dffb640 10 client.16926 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.738+0000 7fcda5d9b640 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb23425580 0x55bb23427970 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9923b8058bb74b8 server_cookie=70969474998042e8 in_seq=0 out_seq=0 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.785+0000 7fcd8dffb640 1 -- 172.21.15.99:0/56118553 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb2330a360 con 0x55bb23425580 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.785+0000 7fcd8dffb640 10 client.16926 handle_command_reply: tid=0 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.785+0000 7fcdaf7ec740 1 client.16926 shutdown 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.785+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 msgr2=0x55bb233bdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.785+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 0x55bb233bdf20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcda0005c40 tx=0x7fcda0031400 comp rx=0 tx=0).stop 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 shutdown_connections 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bb23425580 0x55bb23427970 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcd8dffb640 0 client.16926 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb233a6ad0 0x55bb233bdf20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bb2339c210 0x55bb2341df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 --2- 172.21.15.99:0/56118553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb2339b8f0 0x55bb2341da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 >> 172.21.15.99:0/56118553 conn(0x55bb23397e00 msgr2=0x55bb2339d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 shutdown_connections 2024-05-14T17:17:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 1 -- 172.21.15.99:0/56118553 wait complete. 2024-05-14T17:17:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdaf7ec740 20 client.16926 trim_cache size 0 max 16384 2024-05-14T17:17:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:17:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:17:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcd900b2a10 msgr2=0x7fcd900b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcd900b2a10 0x7fcd900b4ed0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd980e04c0 tx=0x7fcd98000f40 comp rx=0 tx=0).stop 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 msgr2=0x7fcda8199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 0x7fcda8199160 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800ebc0 tx=0x7fcd9800c6a0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 shutdown_connections 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fcd900b2a10 0x7fcd900b4ed0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcda819da20 0x7fcda819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda8107e30 0x7fcda81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 --2- 172.21.15.99:0/3581410121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda81011c0 0x7fcda8199160 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 >> 172.21.15.99:0/3581410121 conn(0x7fcda8077eb0 msgr2=0x7fcda8106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.786+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 shutdown_connections 2024-05-14T17:17:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:02.787+0000 7fcdac9f7640 1 -- 172.21.15.99:0/3581410121 wait complete. 2024-05-14T17:17:03.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:03 smithi099 ceph-mon[31851]: pgmap v421: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-14T17:17:03.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:03 smithi193 ceph-mon[35962]: pgmap v421: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-14T17:17:03.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 523906 3.77 MB/sec execute 555 sec latency 1667.279 ms 2024-05-14T17:17:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:03 smithi150 ceph-mon[36204]: pgmap v421: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-05-14T17:17:04.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 523906 3.76 MB/sec execute 556 sec latency 2667.355 ms 2024-05-14T17:17:05.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:05 smithi099 ceph-mon[31851]: pgmap v422: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-05-14T17:17:05.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:05 smithi193 ceph-mon[35962]: pgmap v422: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-05-14T17:17:05.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 525990 3.78 MB/sec execute 557 sec latency 2708.161 ms 2024-05-14T17:17:05.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:05 smithi150 ceph-mon[36204]: pgmap v422: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-05-14T17:17:06.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:06 smithi099 ceph-mon[31851]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:17:06.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:06 smithi193 ceph-mon[35962]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:17:06.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 527323 3.78 MB/sec execute 558 sec latency 33.731 ms 2024-05-14T17:17:06.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:06 smithi150 ceph-mon[36204]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:17:07.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:07 smithi099 ceph-mon[31851]: pgmap v424: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-05-14T17:17:07.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:07 smithi193 ceph-mon[35962]: pgmap v424: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-05-14T17:17:07.564 INFO:tasks.workunit.client.0.smithi099.stdout: 1 527456 3.78 MB/sec execute 559 sec latency 915.407 ms 2024-05-14T17:17:07.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:07 smithi150 ceph-mon[36204]: pgmap v424: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-05-14T17:17:08.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 527456 3.77 MB/sec execute 560 sec latency 1915.533 ms 2024-05-14T17:17:09.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:09 smithi099 ceph-mon[31851]: pgmap v425: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-05-14T17:17:09.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:09 smithi193 ceph-mon[35962]: pgmap v425: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-05-14T17:17:09.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 527456 3.76 MB/sec execute 561 sec latency 2915.589 ms 2024-05-14T17:17:09.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:09 smithi150 ceph-mon[36204]: pgmap v425: 97 pgs: 97 active+clean; 155 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-05-14T17:17:10.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529681 3.78 MB/sec execute 562 sec latency 2926.456 ms 2024-05-14T17:17:11.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:11 smithi099 ceph-mon[31851]: pgmap v426: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-14T17:17:11.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:11 smithi193 ceph-mon[35962]: pgmap v426: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-14T17:17:11.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.77 MB/sec execute 563 sec latency 789.144 ms 2024-05-14T17:17:11.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:11 smithi150 ceph-mon[36204]: pgmap v426: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-14T17:17:12.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.77 MB/sec execute 564 sec latency 1789.233 ms 2024-05-14T17:17:13.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:13 smithi150 ceph-mon[36204]: pgmap v427: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-14T17:17:13.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:13 smithi099 ceph-mon[31851]: pgmap v427: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-14T17:17:13.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:13 smithi193 ceph-mon[35962]: pgmap v427: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-05-14T17:17:13.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.76 MB/sec execute 565 sec latency 2789.322 ms 2024-05-14T17:17:14.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.75 MB/sec execute 566 sec latency 3789.366 ms 2024-05-14T17:17:15.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:15 smithi099 ceph-mon[31851]: pgmap v428: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:15.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:15 smithi099 ceph-mon[31851]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:17:15.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:15 smithi193 ceph-mon[35962]: pgmap v428: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:15.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:15 smithi193 ceph-mon[35962]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:17:15.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.75 MB/sec execute 567 sec latency 4789.445 ms 2024-05-14T17:17:15.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:15 smithi150 ceph-mon[36204]: pgmap v428: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:15.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:15 smithi150 ceph-mon[36204]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 -- 172.21.15.99:0/1315518682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 msgr2=0x7f4ca8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 --2- 172.21.15.99:0/1315518682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca8102ee0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009930 tx=0x7f4c9002a420 comp rx=0 tx=0).stop 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 -- 172.21.15.99:0/1315518682 shutdown_connections 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 --2- 172.21.15.99:0/1315518682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca8103420 0x7f4ca810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 --2- 172.21.15.99:0/1315518682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca8102ee0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 --2- 172.21.15.99:0/1315518682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ca8108a70 0x7f4ca8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 -- 172.21.15.99:0/1315518682 >> 172.21.15.99:0/1315518682 conn(0x7f4ca8079ee0 msgr2=0x7f4ca807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 -- 172.21.15.99:0/1315518682 shutdown_connections 2024-05-14T17:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.885+0000 7f4cae90a640 1 -- 172.21.15.99:0/1315518682 wait complete. 2024-05-14T17:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 Processor -- start 2024-05-14T17:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 -- start start 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ca8103420 0x7f4ca8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca8108a70 0x7f4ca819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8112d30 con 0x7f4ca8102a60 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8112bb0 con 0x7f4ca8108a70 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cae90a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8112eb0 con 0x7f4ca8103420 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca81991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34050/0 (socket says 172.21.15.99:34050) 2024-05-14T17:17:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4ca7fff640 1 -- 172.21.15.99:0/2121114585 learned_addr learned my addr 172.21.15.99:0/2121114585 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ca8103420 0x7f4ca8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.886+0000 7f4cacea7640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca8108a70 0x7f4ca819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca7fff640 1 -- 172.21.15.99:0/2121114585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ca8103420 msgr2=0x7f4ca8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ca8103420 0x7f4ca8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca7fff640 1 -- 172.21.15.99:0/2121114585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca8108a70 msgr2=0x7f4ca819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca8108a70 0x7f4ca819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca7fff640 1 -- 172.21.15.99:0/2121114585 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca819e1e0 con 0x7f4ca8102a60 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca81991f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800d770 tx=0x7f4c9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca57fa640 1 -- 172.21.15.99:0/2121114585 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98017070 con 0x7f4ca8102a60 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca57fa640 1 -- 172.21.15.99:0/2121114585 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c980054d0 con 0x7f4ca8102a60 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4ca57fa640 1 -- 172.21.15.99:0/2121114585 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98004c30 con 0x7f4ca8102a60 2024-05-14T17:17:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca819e4d0 con 0x7f4ca8102a60 2024-05-14T17:17:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.887+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca81a5d60 con 0x7f4ca8102a60 2024-05-14T17:17:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4ca57fa640 1 -- 172.21.15.99:0/2121114585 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4c98004df0 con 0x7f4ca8102a60 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 Processor -- start 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 -- start start 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98dc8f0 0x5588f98dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98e7ad0 0x5588f98e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588f984fc30 con 0x5588f98dd210 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5588f983f7d0 con 0x5588f98e7ad0 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cb1623740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5588f9843810 con 0x5588f98dc8f0 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98e7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34054/0 (socket says 172.21.15.99:34054) 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4ca7fff640 1 -- 172.21.15.99:0/792358499 learned_addr learned my addr 172.21.15.99:0/792358499 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4cacea7640 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98dc8f0 0x5588f98dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.890+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98e7ad0 0x5588f98e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca57fa640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4c940b2a10 0x7f4c940b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca7fff640 1 -- 172.21.15.99:0/792358499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98dc8f0 msgr2=0x5588f98dccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98dc8f0 0x5588f98dccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca7fff640 1 -- 172.21.15.99:0/792358499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98e7ad0 msgr2=0x5588f98e9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98e7ad0 0x5588f98e9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca7fff640 1 -- 172.21.15.99:0/792358499 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f9554210 con 0x5588f98dd210 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4cacea7640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4c940b2a10 0x7f4c940b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca57fa640 1 -- 172.21.15.99:0/2121114585 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f4c980e1f90 con 0x7f4ca8102a60 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98e7590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4c980df650 tx=0x7f4c980d8880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99c62204d6e1c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4cacea7640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4c940b2a10 0x7f4c940b4ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0005fd0 tx=0x7f4ca0004a40 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4c9e7fc640 1 -- 172.21.15.99:0/792358499 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98020030 con 0x5588f98dd210 2024-05-14T17:17:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4c9e7fc640 1 -- 172.21.15.99:0/792358499 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c98077070 con 0x5588f98dd210 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.891+0000 7f4c9e7fc640 1 -- 172.21.15.99:0/792358499 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980a8c70 con 0x5588f98dd210 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 -- 172.21.15.99:0/792358499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 msgr2=0x5588f98e7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98e7590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4c980df650 tx=0x7f4c980d8880 comp rx=0 tx=0).stop 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 -- 172.21.15.99:0/792358499 shutdown_connections 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98e7ad0 0x5588f98e9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98e7590 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 --2- 172.21.15.99:0/792358499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98dc8f0 0x5588f98dccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 -- 172.21.15.99:0/792358499 >> 172.21.15.99:0/792358499 conn(0x5588f98d8e00 msgr2=0x5588f98d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 -- 172.21.15.99:0/792358499 shutdown_connections 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 -- 172.21.15.99:0/792358499 wait complete. 2024-05-14T17:17:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.892+0000 7f4cb1623740 1 -- start start 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 Processor -- start 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98e7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588f9525790 con 0x5588f98dc8f0 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5588f97fb890 con 0x5588f98dd210 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5588f98ea6e0 con 0x5588f98e7ad0 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cb1623740 10 client.0 _ll_register_callbacks cb 0x5588f97fed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98e7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34058/0 (socket says 172.21.15.99:34058) 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1329742008 learned_addr learned my addr 172.21.15.99:0/1329742008 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4cacea7640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 msgr2=0x5588f98e4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1329742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 msgr2=0x5588f98e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.893+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98e4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1329742008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f9620620 con 0x5588f98dc8f0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cacea7640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98e7460 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4c980df790 tx=0x7f4c980e5e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca93812609bd2744 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4c877fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4c877fe640 1 -- 172.21.15.99:0/1329742008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98020050 con 0x5588f98dc8f0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4c877fe640 1 -- 172.21.15.99:0/1329742008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f4c980050e0 con 0x5588f98dc8f0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c98077070 con 0x5588f98dc8f0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4c877fe640 1 -- 172.21.15.99:0/1329742008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4c980a8a90 con 0x5588f98dc8f0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 10 client.16938 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5588f98f6120 con 0x5588f98dc8f0 2024-05-14T17:17:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4c877fe640 1 -- 172.21.15.99:0/1329742008 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c98077070 con 0x5588f98dc8f0 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4c877fe640 1 -- 172.21.15.99:0/1329742008 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7f4c980a9580 con 0x5588f98dc8f0 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 10 client.16938 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 20 client.16938 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f98f7470 0x5588f98f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 4 client.16938 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.894+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588f9851b80 con 0x5588f98f7470 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.895+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f98f7470 0x5588f98f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.895+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f98f7470 0x5588f98f9860 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8efcec96c2651c2 server_cookie=d35b2f554197d881 in_seq=0 out_seq=0 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.895+0000 7f4c877fe640 10 client.16938 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4c877fe640 1 -- 172.21.15.99:0/1329742008 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5588f9851b80 con 0x5588f98f7470 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4c877fe640 10 client.16938 handle_command_reply: tid=0 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 client.16938 shutdown 2024-05-14T17:17:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 msgr2=0x5588f98e7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98e7460 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4c980df790 tx=0x7f4c980e5e40 comp rx=0 tx=0).stop 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 shutdown_connections 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f98f7470 0x5588f98f9860 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4c877fe640 0 client.16938 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1329742008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98e7460 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.896+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 >> 172.21.15.99:0/1329742008 conn(0x5588f98d8e00 msgr2=0x5588f98d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.897+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 shutdown_connections 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.897+0000 7f4cb1623740 1 -- 172.21.15.99:0/1329742008 wait complete. 2024-05-14T17:17:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.897+0000 7f4cb1623740 20 client.16938 trim_cache size 0 max 16384 2024-05-14T17:17:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 Processor -- start 2024-05-14T17:17:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 -- start start 2024-05-14T17:17:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98d8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81b0f00 con 0x5588f98dc8f0 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ae7a0 con 0x5588f98dd210 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cb1623740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ac610 con 0x5588f98e7ad0 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98d8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34074/0 (socket says 172.21.15.99:34074) 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1745122408 learned_addr learned my addr 172.21.15.99:0/1745122408 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cacea7640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1745122408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 msgr2=0x5588f98e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1745122408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 msgr2=0x5588f98f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 -- 172.21.15.99:0/1745122408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f98d8640 con 0x5588f98dc8f0 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4cacea7640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98d8100 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800d740 tx=0x7f4c980057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8698a4ff04908451 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.912+0000 7f4c9dffb640 1 -- 172.21.15.99:0/1745122408 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98020070 con 0x5588f98dc8f0 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4c9dffb640 1 -- 172.21.15.99:0/1745122408 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c980a8e30 con 0x5588f98dc8f0 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4c9dffb640 1 -- 172.21.15.99:0/1745122408 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980050e0 con 0x5588f98dc8f0 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 -- 172.21.15.99:0/1745122408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 msgr2=0x5588f98d8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98d8100 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800d740 tx=0x7f4c980057e0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 -- 172.21.15.99:0/1745122408 shutdown_connections 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dd210 0x5588f98f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 --2- 172.21.15.99:0/1745122408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dc8f0 0x5588f98d8100 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 -- 172.21.15.99:0/1745122408 >> 172.21.15.99:0/1745122408 conn(0x5588f98d8e00 msgr2=0x5588f98d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 -- 172.21.15.99:0/1745122408 shutdown_connections 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 -- 172.21.15.99:0/1745122408 wait complete. 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.913+0000 7f4cb1623740 1 -- start start 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 Processor -- start 2024-05-14T17:17:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dc8f0 0x5588f98efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81acc60 con 0x5588f98dd210 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ae0d0 con 0x5588f98dc8f0 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ad330 con 0x5588f98e7ad0 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cb1623740 10 client.0 _ll_register_callbacks cb 0x7f4c980ea0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34080/0 (socket says 172.21.15.99:34080) 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 -- 172.21.15.99:0/2217346451 learned_addr learned my addr 172.21.15.99:0/2217346451 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dc8f0 0x5588f98efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cacea7640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 -- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 msgr2=0x5588f98eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98eb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 -- 172.21.15.99:0/2217346451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dc8f0 msgr2=0x5588f98efcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dc8f0 0x5588f98efcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4ca77fe640 1 -- 172.21.15.99:0/2217346451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f98ebbe0 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.914+0000 7f4cacea7640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4ca77fe640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98eb060 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4c90000c00 tx=0x7f4c9002ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43eb5aea918f938d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4c877fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4c877fe640 1 -- 172.21.15.99:0/2217346451 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9004a070 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4c877fe640 1 -- 172.21.15.99:0/2217346451 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f4c900314a0 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c980a8e30 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4c877fe640 1 -- 172.21.15.99:0/2217346451 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4c90056050 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 10 client.16944 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5588f9961830 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4c877fe640 1 -- 172.21.15.99:0/2217346451 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c980a8e30 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4c877fe640 1 -- 172.21.15.99:0/2217346451 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7f4c90003680 con 0x5588f98dd210 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 10 client.16944 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 20 client.16944 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f9961fe0 0x5588f9964400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 4 client.16944 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:17:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.915+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588f984b360 con 0x5588f9961fe0 2024-05-14T17:17:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.916+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f9961fe0 0x5588f9964400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.916+0000 7f4ca7fff640 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f9961fe0 0x5588f9964400 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b660d3f644919ad server_cookie=4e14c686b3998d11 in_seq=0 out_seq=0 2024-05-14T17:17:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.916+0000 7f4c877fe640 10 client.16944 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4c877fe640 1 -- 172.21.15.99:0/2217346451 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5588f984b360 con 0x5588f9961fe0 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4c877fe640 10 client.16944 handle_command_reply: tid=0 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 client.16944 shutdown 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 msgr2=0x5588f98eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98eb060 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4c90000c00 tx=0x7f4c9002ab70 comp rx=0 tx=0).stop 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 shutdown_connections 2024-05-14T17:17:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5588f9961fe0 0x5588f9964400 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588f98e7ad0 0x5588f98eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588f98dd210 0x5588f98eb060 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4c877fe640 0 client.16944 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 --2- 172.21.15.99:0/2217346451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588f98dc8f0 0x5588f98efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 >> 172.21.15.99:0/2217346451 conn(0x5588f98d8e00 msgr2=0x5588f98d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 shutdown_connections 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 1 -- 172.21.15.99:0/2217346451 wait complete. 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.962+0000 7f4cb1623740 20 client.16944 trim_cache size 0 max 16384 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:17:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:17:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4c940b2a10 msgr2=0x7f4c940b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4c940b2a10 0x7f4c940b4ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0005fd0 tx=0x7f4ca0004a40 comp rx=0 tx=0).stop 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 msgr2=0x7f4ca81991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca81991f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800d770 tx=0x7f4c9800dc30 comp rx=0 tx=0).stop 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 shutdown_connections 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4c940b2a10 0x7f4c940b4ed0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca8108a70 0x7f4ca819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ca8103420 0x7f4ca8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 --2- 172.21.15.99:0/2121114585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ca8102a60 0x7f4ca81991f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 >> 172.21.15.99:0/2121114585 conn(0x7f4ca8079ee0 msgr2=0x7f4ca8100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 shutdown_connections 2024-05-14T17:17:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:15.963+0000 7f4cae90a640 1 -- 172.21.15.99:0/2121114585 wait complete. 2024-05-14T17:17:16.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.74 MB/sec execute 568 sec latency 5789.574 ms 2024-05-14T17:17:17.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:17 smithi099 ceph-mon[31851]: pgmap v430: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:17.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:17 smithi193 ceph-mon[35962]: pgmap v430: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:17.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.73 MB/sec execute 569 sec latency 6789.653 ms 2024-05-14T17:17:17.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:17 smithi150 ceph-mon[36204]: pgmap v430: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:18.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.73 MB/sec execute 570 sec latency 7789.756 ms 2024-05-14T17:17:19.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:19 smithi099 ceph-mon[31851]: pgmap v431: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:19.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:19 smithi193 ceph-mon[35962]: pgmap v431: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:19.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 529902 3.72 MB/sec execute 571 sec latency 8789.793 ms 2024-05-14T17:17:19.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:19 smithi150 ceph-mon[36204]: pgmap v431: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-05-14T17:17:20.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 530999 3.72 MB/sec execute 572 sec latency 8830.027 ms 2024-05-14T17:17:21.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:21 smithi099 ceph-mon[31851]: pgmap v432: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-05-14T17:17:21.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:21 smithi193 ceph-mon[35962]: pgmap v432: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-05-14T17:17:21.565 INFO:tasks.workunit.client.0.smithi099.stdout: 1 530999 3.72 MB/sec execute 573 sec latency 1383.716 ms 2024-05-14T17:17:21.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:21 smithi150 ceph-mon[36204]: pgmap v432: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-05-14T17:17:22.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 530999 3.71 MB/sec execute 574 sec latency 2383.807 ms 2024-05-14T17:17:23.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:23 smithi099 ceph-mon[31851]: pgmap v433: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-05-14T17:17:23.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:23 smithi193 ceph-mon[35962]: pgmap v433: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-05-14T17:17:23.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 530999 3.70 MB/sec execute 575 sec latency 3383.919 ms 2024-05-14T17:17:23.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:23 smithi150 ceph-mon[36204]: pgmap v433: 97 pgs: 97 active+clean; 157 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-05-14T17:17:24.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 530999 3.70 MB/sec execute 576 sec latency 4383.966 ms 2024-05-14T17:17:25.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:25 smithi099 ceph-mon[31851]: pgmap v434: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 68 op/s 2024-05-14T17:17:25.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:25 smithi193 ceph-mon[35962]: pgmap v434: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 68 op/s 2024-05-14T17:17:25.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 533212 3.71 MB/sec execute 577 sec latency 4424.879 ms 2024-05-14T17:17:25.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:25 smithi150 ceph-mon[36204]: pgmap v434: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 68 op/s 2024-05-14T17:17:26.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 534579 3.72 MB/sec execute 578 sec latency 87.596 ms 2024-05-14T17:17:27.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:27 smithi099 ceph-mon[31851]: pgmap v435: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 970 KiB/s wr, 63 op/s 2024-05-14T17:17:27.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:27 smithi193 ceph-mon[35962]: pgmap v435: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 970 KiB/s wr, 63 op/s 2024-05-14T17:17:27.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 534579 3.71 MB/sec execute 579 sec latency 1087.679 ms 2024-05-14T17:17:27.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:27 smithi150 ceph-mon[36204]: pgmap v435: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 970 KiB/s wr, 63 op/s 2024-05-14T17:17:28.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 534579 3.70 MB/sec execute 580 sec latency 2087.751 ms 2024-05-14T17:17:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.059+0000 7f3be8cdc640 1 -- 172.21.15.99:0/1547506866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 msgr2=0x7f3be410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.059+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/1547506866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be410f930 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009930 tx=0x7f3bdc02a420 comp rx=0 tx=0).stop 2024-05-14T17:17:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 -- 172.21.15.99:0/1547506866 shutdown_connections 2024-05-14T17:17:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/1547506866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be410f930 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/1547506866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4102a60 0x7f3be4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/1547506866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4108a70 0x7f3be4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 -- 172.21.15.99:0/1547506866 >> 172.21.15.99:0/1547506866 conn(0x7f3be4079ee0 msgr2=0x7f3be407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 -- 172.21.15.99:0/1547506866 shutdown_connections 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 -- 172.21.15.99:0/1547506866 wait complete. 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 Processor -- start 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.060+0000 7f3be8cdc640 1 -- start start 2024-05-14T17:17:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4102a60 0x7f3be4074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be4075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4108a70 0x7f3be40757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112d30 con 0x7f3be4103420 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112bb0 con 0x7f3be4102a60 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112eb0 con 0x7f3be4108a70 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be4075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be4075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32970/0 (socket says 172.21.15.99:32970) 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 -- 172.21.15.99:0/3327148340 learned_addr learned my addr 172.21.15.99:0/3327148340 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4108a70 0x7f3be40757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 -- 172.21.15.99:0/3327148340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4108a70 msgr2=0x7f3be40757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be259c640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4102a60 0x7f3be4074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4108a70 0x7f3be40757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 -- 172.21.15.99:0/3327148340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4102a60 msgr2=0x7f3be4074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4102a60 0x7f3be4074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 -- 172.21.15.99:0/3327148340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be41a7630 con 0x7f3be4103420 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4108a70 0x7f3be40757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be4075290 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800d770 tx=0x7f3bd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3bd37fe640 1 -- 172.21.15.99:0/3327148340 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8017070 con 0x7f3be4103420 2024-05-14T17:17:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be41a7920 con 0x7f3be4103420 2024-05-14T17:17:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3bd37fe640 1 -- 172.21.15.99:0/3327148340 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd80054d0 con 0x7f3be4103420 2024-05-14T17:17:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.061+0000 7f3bd37fe640 1 -- 172.21.15.99:0/3327148340 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8004c30 con 0x7f3be4103420 2024-05-14T17:17:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.062+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41a7e70 con 0x7f3be4103420 2024-05-14T17:17:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 Processor -- start 2024-05-14T17:17:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3bd37fe640 1 -- 172.21.15.99:0/3327148340 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3bd8004dd0 con 0x7f3be4103420 2024-05-14T17:17:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 -- start start 2024-05-14T17:17:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 0x55890403fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558904040210 0x55890404a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x55890404cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558903fb2c30 con 0x55890403f8f0 2024-05-14T17:17:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558903fa27d0 con 0x55890404aad0 2024-05-14T17:17:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.064+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558903fa6810 con 0x558904040210 2024-05-14T17:17:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 0x55890403fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 0x55890403fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32980/0 (socket says 172.21.15.99:32980) 2024-05-14T17:17:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558904040210 0x55890404a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 -- 172.21.15.99:0/1052622392 learned_addr learned my addr 172.21.15.99:0/1052622392 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x55890404cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 -- 172.21.15.99:0/1052622392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558904040210 msgr2=0x55890404a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558904040210 0x55890404a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 -- 172.21.15.99:0/1052622392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 msgr2=0x55890404cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x55890404cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 -- 172.21.15.99:0/1052622392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558903cb7210 con 0x55890403f8f0 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be259c640 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558904040210 0x55890404a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x55890404cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3bd37fe640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3bcc0b2a40 0x7f3bcc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 0x55890403fcd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc02a940 tx=0x7f3bdc032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d636e921052eb2c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3be259c640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3bcc0b2a40 0x7f3bcc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3bd37fe640 1 -- 172.21.15.99:0/3327148340 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f3bd80e20f0 con 0x7f3be4103420 2024-05-14T17:17:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3bd0ff9640 1 -- 172.21.15.99:0/1052622392 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc03f070 con 0x55890403f8f0 2024-05-14T17:17:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3bd0ff9640 1 -- 172.21.15.99:0/1052622392 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bdc0316a0 con 0x55890403f8f0 2024-05-14T17:17:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.065+0000 7f3bd0ff9640 1 -- 172.21.15.99:0/1052622392 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc035070 con 0x55890403f8f0 2024-05-14T17:17:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1052622392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 msgr2=0x55890403fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3be259c640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3bcc0b2a40 0x7f3bcc0b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x558904040e90 tx=0x7f3bd400b5c0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 0x55890403fcd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc02a940 tx=0x7f3bdc032dc0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1052622392 shutdown_connections 2024-05-14T17:17:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x55890404cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558904040210 0x55890404a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1052622392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890403f8f0 0x55890403fcd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1052622392 >> 172.21.15.99:0/1052622392 conn(0x55890403be00 msgr2=0x55890403c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1052622392 shutdown_connections 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.066+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1052622392 wait complete. 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 -- start start 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 Processor -- start 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x55890404a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558904040210 0x5589040476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 0x558904047c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558903c88790 con 0x55890404aad0 2024-05-14T17:17:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558903f5e890 con 0x558904040210 2024-05-14T17:17:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55890404d740 con 0x55890403f8f0 2024-05-14T17:17:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3beb9f5740 10 client.0 _ll_register_callbacks cb 0x558903f61d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 0x558904047c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 0x558904047c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32994/0 (socket says 172.21.15.99:32994) 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 -- 172.21.15.99:0/216495021 learned_addr learned my addr 172.21.15.99:0/216495021 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be259c640 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x55890404a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 -- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 msgr2=0x55890404a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x55890404a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 -- 172.21.15.99:0/216495021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558904040210 msgr2=0x5589040476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558904040210 0x5589040476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.067+0000 7f3be2d9d640 1 -- 172.21.15.99:0/216495021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558903d83620 con 0x55890404aad0 2024-05-14T17:17:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 0x558904047c00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc02a460 tx=0x7f3bdc032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd17dd499f2648ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/216495021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc03f050 con 0x55890404aad0 2024-05-14T17:17:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/216495021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f3bdc004240 con 0x55890404aad0 2024-05-14T17:17:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bdc0316a0 con 0x55890404aad0 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/216495021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc035040 con 0x55890404aad0 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 10 client.16956 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558904059100 con 0x55890404aad0 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/216495021 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bdc0316a0 con 0x55890404aad0 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/216495021 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7f3bdc04d470 con 0x55890404aad0 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 10 client.16956 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 20 client.16956 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55890405a450 0x55890405c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 4 client.16956 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:17:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.068+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558903fb4b80 con 0x55890405a450 2024-05-14T17:17:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.069+0000 7f3be259c640 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55890405a450 0x55890405c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.069+0000 7f3be259c640 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55890405a450 0x55890405c840 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec45564bc7473f1 server_cookie=7efe17693b6f8b81 in_seq=0 out_seq=0 2024-05-14T17:17:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.069+0000 7f3bc9ffb640 10 client.16956 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/216495021 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558903fb4b80 con 0x55890405a450 2024-05-14T17:17:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3bc9ffb640 10 client.16956 handle_command_reply: tid=0 2024-05-14T17:17:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 client.16956 shutdown 2024-05-14T17:17:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 msgr2=0x558904047c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 0x558904047c00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc02a460 tx=0x7f3bdc032c10 comp rx=0 tx=0).stop 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 shutdown_connections 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55890405a450 0x55890405c840 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890404aad0 0x558904047c00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558904040210 0x5589040476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3bc9ffb640 0 client.16956 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/216495021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x55890404a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 >> 172.21.15.99:0/216495021 conn(0x55890403be00 msgr2=0x55890403b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 shutdown_connections 2024-05-14T17:17:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 1 -- 172.21.15.99:0/216495021 wait complete. 2024-05-14T17:17:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.070+0000 7f3beb9f5740 20 client.16956 trim_cache size 0 max 16384 2024-05-14T17:17:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.085+0000 7f3beb9f5740 1 Processor -- start 2024-05-14T17:17:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.085+0000 7f3beb9f5740 1 -- start start 2024-05-14T17:17:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904056610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904061de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904062320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41b30b0 con 0x558904040210 2024-05-14T17:17:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41b0950 con 0x55890404aad0 2024-05-14T17:17:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ae7c0 con 0x55890403f8f0 2024-05-14T17:17:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904061de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904061de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32996/0 (socket says 172.21.15.99:32996) 2024-05-14T17:17:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 -- 172.21.15.99:0/2734726579 learned_addr learned my addr 172.21.15.99:0/2734726579 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be259c640 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904056610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 -- 172.21.15.99:0/2734726579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 msgr2=0x558904056610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904056610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 -- 172.21.15.99:0/2734726579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 msgr2=0x558904062320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904062320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 -- 172.21.15.99:0/2734726579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890405cbb0 con 0x558904040210 2024-05-14T17:17:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be259c640 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904056610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904061de0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800d740 tx=0x7f3bd80a8be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12b8ea280972565 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3bcbfff640 1 -- 172.21.15.99:0/2734726579 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8020050 con 0x558904040210 2024-05-14T17:17:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3bcbfff640 1 -- 172.21.15.99:0/2734726579 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd80a9b70 con 0x558904040210 2024-05-14T17:17:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.086+0000 7f3bcbfff640 1 -- 172.21.15.99:0/2734726579 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80a5700 con 0x558904040210 2024-05-14T17:17:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 -- 172.21.15.99:0/2734726579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 msgr2=0x558904061de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904061de0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800d740 tx=0x7f3bd80a8be0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 -- 172.21.15.99:0/2734726579 shutdown_connections 2024-05-14T17:17:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904062320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904061de0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/2734726579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904056610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 -- 172.21.15.99:0/2734726579 >> 172.21.15.99:0/2734726579 conn(0x55890403be00 msgr2=0x55890403b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 -- 172.21.15.99:0/2734726579 shutdown_connections 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 -- 172.21.15.99:0/2734726579 wait complete. 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 -- start start 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 Processor -- start 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.087+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904052da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904050130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904050670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41aee10 con 0x558904040210 2024-05-14T17:17:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41b0280 con 0x55890404aad0 2024-05-14T17:17:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41af4e0 con 0x55890403f8f0 2024-05-14T17:17:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 10 client.0 _ll_register_callbacks cb 0x7f3bdc02ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904050130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904050130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33008/0 (socket says 172.21.15.99:33008) 2024-05-14T17:17:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 -- 172.21.15.99:0/1708910393 learned_addr learned my addr 172.21.15.99:0/1708910393 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be259c640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904052da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904050670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 -- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 msgr2=0x558904052da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904052da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 -- 172.21.15.99:0/1708910393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 msgr2=0x558904050670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904050670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 -- 172.21.15.99:0/1708910393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558904050d10 con 0x558904040210 2024-05-14T17:17:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be2d9d640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904050670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:17:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3be1d9b640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904050130 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80a54f0 tx=0x7f3bd80dfe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36333f24e420ea97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/1708910393 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80ad820 con 0x558904040210 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/1708910393 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f3bd80b1e40 con 0x558904040210 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd80a9b70 con 0x558904040210 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/1708910393 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3bd8020030 con 0x558904040210 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 10 client.16962 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5589040c47d0 con 0x558904040210 2024-05-14T17:17:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.088+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/1708910393 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd80a9b70 con 0x558904040210 2024-05-14T17:17:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/1708910393 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7f3bd80a3b90 con 0x558904040210 2024-05-14T17:17:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3beb9f5740 10 client.16962 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3beb9f5740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3beb9f5740 20 client.16962 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5589040c4e90 0x5589040c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3beb9f5740 4 client.16962 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:17:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558903fae360 con 0x5589040c4e90 2024-05-14T17:17:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3be259c640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5589040c4e90 0x5589040c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3bc9ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.089+0000 7f3be259c640 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5589040c4e90 0x5589040c7280 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955508fc2e303336 server_cookie=3b4736de13aa77ec in_seq=0 out_seq=0 2024-05-14T17:17:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.132+0000 7f3bc9ffb640 1 -- 172.21.15.99:0/1708910393 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558903fae360 con 0x5589040c4e90 2024-05-14T17:17:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.132+0000 7f3bc9ffb640 10 client.16962 handle_command_reply: tid=0 2024-05-14T17:17:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.132+0000 7f3beb9f5740 1 client.16962 shutdown 2024-05-14T17:17:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 msgr2=0x558904050130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904050130 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80a54f0 tx=0x7f3bd80dfe10 comp rx=0 tx=0).stop 2024-05-14T17:17:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 shutdown_connections 2024-05-14T17:17:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5589040c4e90 0x5589040c7280 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55890404aad0 0x558904050670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558904040210 0x558904050130 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3bc9ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 --2- 172.21.15.99:0/1708910393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55890403f8f0 0x558904052da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 >> 172.21.15.99:0/1708910393 conn(0x55890403be00 msgr2=0x558904055600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 shutdown_connections 2024-05-14T17:17:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 1 -- 172.21.15.99:0/1708910393 wait complete. 2024-05-14T17:17:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3beb9f5740 20 client.16962 trim_cache size 0 max 16384 2024-05-14T17:17:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:17:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:17:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:17:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3bcc0b2a40 msgr2=0x7f3bcc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3bcc0b2a40 0x7f3bcc0b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x558904040e90 tx=0x7f3bd400b5c0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 msgr2=0x7f3be4075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be4075290 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800d770 tx=0x7f3bd800dc30 comp rx=0 tx=0).stop 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 shutdown_connections 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f3bcc0b2a40 0x7f3bcc0b4f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be4108a70 0x7f3be40757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3be4103420 0x7f3be4075290 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 --2- 172.21.15.99:0/3327148340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3be4102a60 0x7f3be4074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.133+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 >> 172.21.15.99:0/3327148340 conn(0x7f3be4079ee0 msgr2=0x7f3be4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.134+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 shutdown_connections 2024-05-14T17:17:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:29.134+0000 7f3be8cdc640 1 -- 172.21.15.99:0/3327148340 wait complete. 2024-05-14T17:17:29.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:29 smithi099 ceph-mon[31851]: pgmap v436: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 878 KiB/s wr, 57 op/s 2024-05-14T17:17:29.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:29 smithi193 ceph-mon[35962]: pgmap v436: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 878 KiB/s wr, 57 op/s 2024-05-14T17:17:29.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 534579 3.70 MB/sec execute 581 sec latency 3087.783 ms 2024-05-14T17:17:29.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:29 smithi150 ceph-mon[36204]: pgmap v436: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 878 KiB/s wr, 57 op/s 2024-05-14T17:17:30.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 536801 3.72 MB/sec execute 582 sec latency 3094.662 ms 2024-05-14T17:17:31.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:31 smithi099 ceph-mon[31851]: pgmap v437: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-14T17:17:31.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:31 smithi193 ceph-mon[35962]: pgmap v437: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-14T17:17:31.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 538122 3.72 MB/sec execute 583 sec latency 230.390 ms 2024-05-14T17:17:31.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:31 smithi150 ceph-mon[36204]: pgmap v437: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-05-14T17:17:32.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 538122 3.71 MB/sec execute 584 sec latency 1230.493 ms 2024-05-14T17:17:33.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:33 smithi099 ceph-mon[31851]: pgmap v438: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-05-14T17:17:33.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:33 smithi193 ceph-mon[35962]: pgmap v438: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-05-14T17:17:33.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 538122 3.71 MB/sec execute 585 sec latency 2230.610 ms 2024-05-14T17:17:33.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:33 smithi150 ceph-mon[36204]: pgmap v438: 97 pgs: 97 active+clean; 161 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-05-14T17:17:34.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 538122 3.70 MB/sec execute 586 sec latency 3230.659 ms 2024-05-14T17:17:35.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:35 smithi193 ceph-mon[35962]: pgmap v439: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-05-14T17:17:35.566 INFO:tasks.workunit.client.0.smithi099.stdout: 1 540345 3.72 MB/sec execute 587 sec latency 3270.151 ms 2024-05-14T17:17:35.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:35 smithi150 ceph-mon[36204]: pgmap v439: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-05-14T17:17:35.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:35 smithi099 ceph-mon[31851]: pgmap v439: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-05-14T17:17:36.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 541671 3.72 MB/sec execute 588 sec latency 193.683 ms 2024-05-14T17:17:37.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:37 smithi193 ceph-mon[35962]: pgmap v440: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-05-14T17:17:37.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 541671 3.71 MB/sec execute 589 sec latency 1193.792 ms 2024-05-14T17:17:37.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:37 smithi150 ceph-mon[36204]: pgmap v440: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-05-14T17:17:37.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:37 smithi099 ceph-mon[31851]: pgmap v440: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-05-14T17:17:38.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 541671 3.71 MB/sec execute 590 sec latency 2193.910 ms 2024-05-14T17:17:39.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:39 smithi193 ceph-mon[35962]: pgmap v441: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-05-14T17:17:39.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 541671 3.70 MB/sec execute 591 sec latency 3193.976 ms 2024-05-14T17:17:39.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:39 smithi150 ceph-mon[36204]: pgmap v441: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-05-14T17:17:39.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:39 smithi099 ceph-mon[31851]: pgmap v441: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-05-14T17:17:40.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 543897 3.72 MB/sec execute 592 sec latency 3241.725 ms 2024-05-14T17:17:41.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:41 smithi193 ceph-mon[35962]: pgmap v442: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-05-14T17:17:41.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 545212 3.72 MB/sec execute 593 sec latency 200.085 ms 2024-05-14T17:17:41.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:41 smithi150 ceph-mon[36204]: pgmap v442: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-05-14T17:17:41.763 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:41 smithi099 ceph-mon[31851]: pgmap v442: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.228+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1119482300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac106920 msgr2=0x7fc1ac106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.228+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1119482300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac106920 0x7fc1ac106d00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009900 tx=0x7fc1a802a410 comp rx=0 tx=0).stop 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1119482300 shutdown_connections 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1119482300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1119482300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 0x7fc1ac1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1119482300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac106920 0x7fc1ac106d00 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1119482300 >> 172.21.15.99:0/1119482300 conn(0x7fc1ac078730 msgr2=0x7fc1ac078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1119482300 shutdown_connections 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1119482300 wait complete. 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 Processor -- start 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.229+0000 7fc1b3e45640 1 -- start start 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 0x7fc1ac0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac0ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1ac106920 0x7fc1ac1001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac111bc0 con 0x7fc1ac1025f0 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac111a40 con 0x7fc1ac106920 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac111d40 con 0x7fc1ac101c30 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac0ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac0ffc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59196/0 (socket says 172.21.15.99:59196) 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1538287380 learned_addr learned my addr 172.21.15.99:0/1538287380 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 0x7fc1ac0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1ac106920 0x7fc1ac1001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1538287380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 msgr2=0x7fc1ac0ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 0x7fc1ac0ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1538287380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1ac106920 msgr2=0x7fc1ac1001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1ac106920 0x7fc1ac1001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1538287380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac1a7600 con 0x7fc1ac1025f0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 0x7fc1ac0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac0ffc90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00d770 tx=0x7fc19c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1a2ffd640 1 -- 172.21.15.99:0/1538287380 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1ac1025f0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1a2ffd640 1 -- 172.21.15.99:0/1538287380 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c0054d0 con 0x7fc1ac1025f0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac1a78f0 con 0x7fc1ac1025f0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1a2ffd640 1 -- 172.21.15.99:0/1538287380 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c004c30 con 0x7fc1ac1025f0 2024-05-14T17:17:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.230+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a7df0 con 0x7fc1ac1025f0 2024-05-14T17:17:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.233+0000 7fc1a2ffd640 1 -- 172.21.15.99:0/1538287380 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc19c004e10 con 0x7fc1ac1025f0 2024-05-14T17:17:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.233+0000 7fc1b6c39740 1 Processor -- start 2024-05-14T17:17:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 -- start start 2024-05-14T17:17:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07962cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 0x561b0796d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b0796dad0 0x561b0796fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561b078d5c30 con 0x561b0796dad0 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b078c57d0 con 0x561b07963210 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561b078c9810 con 0x561b079628f0 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b0796dad0 0x561b0796fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 0x561b0796d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 0x561b0796d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.99:60144/0 (socket says 172.21.15.99:60144) 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b0796dad0 0x561b0796fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59208/0 (socket says 172.21.15.99:59208) 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 -- 172.21.15.99:0/4186913093 learned_addr learned my addr 172.21.15.99:0/4186913093 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07962cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1a2ffd640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc1980b2a10 0x7fc1980b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 -- 172.21.15.99:0/4186913093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 msgr2=0x561b07962cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc1980b2a10 0x7fc1980b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07962cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 -- 172.21.15.99:0/4186913093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b0796dad0 msgr2=0x561b0796fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b0796dad0 0x561b0796fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1b1be1640 1 -- 172.21.15.99:0/4186913093 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b075da210 con 0x561b07963210 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.234+0000 7fc1a2ffd640 1 -- 172.21.15.99:0/1538287380 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fc19c0e2140 con 0x7fc1ac1025f0 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.235+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 0x561b0796d590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009900 tx=0x7fc1a8032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f855f68bc6165df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.235+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc1980b2a10 0x7fc1980b4ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009fd0 tx=0x7fc1a4004ab0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.235+0000 7fc197fff640 1 -- 172.21.15.99:0/4186913093 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a803f070 con 0x561b07963210 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.235+0000 7fc197fff640 1 -- 172.21.15.99:0/4186913093 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a8032e10 con 0x561b07963210 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.235+0000 7fc197fff640 1 -- 172.21.15.99:0/4186913093 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8035070 con 0x561b07963210 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 -- 172.21.15.99:0/4186913093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 msgr2=0x561b0796d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 0x561b0796d590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009900 tx=0x7fc1a8032c20 comp rx=0 tx=0).stop 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 -- 172.21.15.99:0/4186913093 shutdown_connections 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b0796dad0 0x561b0796fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b07963210 0x561b0796d590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/4186913093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07962cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 -- 172.21.15.99:0/4186913093 >> 172.21.15.99:0/4186913093 conn(0x561b0795ee00 msgr2=0x561b0795f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 -- 172.21.15.99:0/4186913093 shutdown_connections 2024-05-14T17:17:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 -- 172.21.15.99:0/4186913093 wait complete. 2024-05-14T17:17:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 -- start start 2024-05-14T17:17:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.236+0000 7fc1b6c39740 1 Processor -- start 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b0796d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b0796ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561b075ab790 con 0x561b07963210 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561b07881890 con 0x561b079628f0 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561b07970740 con 0x561b0796dad0 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b6c39740 10 client.0 _ll_register_callbacks cb 0x561b07884d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59224/0 (socket says 172.21.15.99:59224) 2024-05-14T17:17:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 -- 172.21.15.99:0/2898310222 learned_addr learned my addr 172.21.15.99:0/2898310222 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b0796ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b0796d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 -- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 msgr2=0x561b0796ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b0796ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 -- 172.21.15.99:0/2898310222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 msgr2=0x561b0796d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b0796d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 -- 172.21.15.99:0/2898310222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b076a6620 con 0x561b07963210 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b0796d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796a710 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0a93b0 tx=0x7fc19c0a8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b767912f0361317 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1957fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1957fa640 1 -- 172.21.15.99:0/2898310222 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c020070 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.237+0000 7fc1957fa640 1 -- 172.21.15.99:0/2898310222 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fc19c0b1ff0 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a8032e10 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1957fa640 1 -- 172.21.15.99:0/2898310222 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc19c005680 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 10 client.16974 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561b0797c1c0 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1957fa640 1 -- 172.21.15.99:0/2898310222 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a8032e10 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1957fa640 1 -- 172.21.15.99:0/2898310222 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7fc19c0a3b40 con 0x561b07963210 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 10 client.16974 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 20 client.16974 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b0797d680 0x561b0797fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 4 client.16974 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b078d7b80 con 0x561b0797d680 2024-05-14T17:17:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.238+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b0797d680 0x561b0797fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.239+0000 7fc1957fa640 10 client.16974 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.239+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b0797d680 0x561b0797fa70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=256112e5fabbd220 server_cookie=4ecaa67579c395e2 in_seq=0 out_seq=0 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.239+0000 7fc1957fa640 1 -- 172.21.15.99:0/2898310222 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b078d7b80 con 0x561b0797d680 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.239+0000 7fc1957fa640 10 client.16974 handle_command_reply: tid=0 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.239+0000 7fc1b6c39740 1 client.16974 shutdown 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.239+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 msgr2=0x561b0796a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796a710 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0a93b0 tx=0x7fc19c0a8d70 comp rx=0 tx=0).stop 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 shutdown_connections 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b0797d680 0x561b0797fa70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b0796ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796a710 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1957fa640 0 client.16974 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/2898310222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b0796d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 >> 172.21.15.99:0/2898310222 conn(0x561b0795ee00 msgr2=0x561b0795e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 shutdown_connections 2024-05-14T17:17:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 1 -- 172.21.15.99:0/2898310222 wait complete. 2024-05-14T17:17:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.240+0000 7fc1b6c39740 20 client.16974 trim_cache size 0 max 16384 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 Processor -- start 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 -- start start 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b07979610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b07985380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1b2fe0 con 0x561b07963210 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1b0880 con 0x561b079628f0 2024-05-14T17:17:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1ae6f0 con 0x561b0796dad0 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59234/0 (socket says 172.21.15.99:59234) 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.255+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1153185344 learned_addr learned my addr 172.21.15.99:0/1153185344 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b07979610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b07985380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1153185344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 msgr2=0x561b07985380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b07985380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1153185344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 msgr2=0x561b07979610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b07979610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1153185344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0796afe0 con 0x561b07963210 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796cf50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0a9360 tx=0x7fc19c0910f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489b95041a3b703b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1977fe640 1 -- 172.21.15.99:0/1153185344 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c020070 con 0x561b07963210 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1977fe640 1 -- 172.21.15.99:0/1153185344 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c0a9b20 con 0x561b07963210 2024-05-14T17:17:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1977fe640 1 -- 172.21.15.99:0/1153185344 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0b2050 con 0x561b07963210 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1153185344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 msgr2=0x561b0796cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.256+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796cf50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0a9360 tx=0x7fc19c0910f0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1153185344 shutdown_connections 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b0796dad0 0x561b07985380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b0796cf50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1153185344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b079628f0 0x561b07979610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1153185344 >> 172.21.15.99:0/1153185344 conn(0x561b0795ee00 msgr2=0x561b07964b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1153185344 shutdown_connections 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1153185344 wait complete. 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- start start 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 Processor -- start 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07975ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b079671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b0796dad0 0x561b07967720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1aed40 con 0x561b07963210 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1b01b0 con 0x561b0796dad0 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1af410 con 0x561b079628f0 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b6c39740 10 client.0 _ll_register_callbacks cb 0x7fc19c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b079671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.257+0000 7fc1b13e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b079671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59238/0 (socket says 172.21.15.99:59238) 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1868478923 learned_addr learned my addr 172.21.15.99:0/1868478923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b23e2640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b0796dad0 0x561b07967720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07975ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 msgr2=0x561b07975ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07975ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1868478923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b0796dad0 msgr2=0x561b07967720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b0796dad0 0x561b07967720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 -- 172.21.15.99:0/1868478923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b07967f20 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b13e0640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b079671e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0a93b0 tx=0x7fc19c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7d74a98ef9339e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1957fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1957fa640 1 -- 172.21.15.99:0/1868478923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c020070 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1957fa640 1 -- 172.21.15.99:0/1868478923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fc19c0a4d90 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c0a9b20 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1957fa640 1 -- 172.21.15.99:0/1868478923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc19c0a8b90 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b6c39740 10 client.16980 fetch_fsmap learned FSMap version 43 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561b079e7f10 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.258+0000 7fc1957fa640 1 -- 172.21.15.99:0/1868478923 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c0a9b20 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1957fa640 1 -- 172.21.15.99:0/1868478923 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 43) ==== 8342+0+0 (secure 0 0 0) 0x7fc19c0ae960 con 0x561b07963210 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b6c39740 10 client.16980 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b6c39740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b6c39740 20 client.16980 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b079e8390 0x561b079ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b6c39740 4 client.16980 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:17:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b078d1360 con 0x561b079e8390 2024-05-14T17:17:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b079e8390 0x561b079ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1b1be1640 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b079e8390 0x561b079ea780 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db88b7046b2fa67a server_cookie=3c94f243bb079519 in_seq=0 out_seq=0 2024-05-14T17:17:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.259+0000 7fc1957fa640 10 client.16980 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.302+0000 7fc1957fa640 1 -- 172.21.15.99:0/1868478923 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561b078d1360 con 0x561b079e8390 2024-05-14T17:17:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.302+0000 7fc1957fa640 10 client.16980 handle_command_reply: tid=0 2024-05-14T17:17:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.302+0000 7fc1b6c39740 1 client.16980 shutdown 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.302+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 msgr2=0x561b079671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.302+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b079671e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0a93b0 tx=0x7fc19c009f90 comp rx=0 tx=0).stop 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.302+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 shutdown_connections 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x561b079e8390 0x561b079ea780 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b0796dad0 0x561b07967720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1957fa640 0 client.16980 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b07963210 0x561b079671e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 --2- 172.21.15.99:0/1868478923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561b079628f0 0x561b07975ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 >> 172.21.15.99:0/1868478923 conn(0x561b0795ee00 msgr2=0x561b079642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 shutdown_connections 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 1 -- 172.21.15.99:0/1868478923 wait complete. 2024-05-14T17:17:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b6c39740 20 client.16980 trim_cache size 0 max 16384 2024-05-14T17:17:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:17:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:17:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:17:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc1980b2a10 msgr2=0x7fc1980b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc1980b2a10 0x7fc1980b4ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009fd0 tx=0x7fc1a4004ab0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 msgr2=0x7fc1ac0ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac0ffc90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00d770 tx=0x7fc19c00dc30 comp rx=0 tx=0).stop 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.303+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 shutdown_connections 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fc1980b2a10 0x7fc1980b4ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1ac106920 0x7fc1ac1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac1025f0 0x7fc1ac0ffc90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 --2- 172.21.15.99:0/1538287380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ac101c30 0x7fc1ac0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 >> 172.21.15.99:0/1538287380 conn(0x7fc1ac078730 msgr2=0x7fc1ac06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 shutdown_connections 2024-05-14T17:17:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:42.304+0000 7fc1b3e45640 1 -- 172.21.15.99:0/1538287380 wait complete. 2024-05-14T17:17:42.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 545212 3.71 MB/sec execute 594 sec latency 1200.202 ms 2024-05-14T17:17:43.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:43 smithi193 ceph-mon[35962]: pgmap v443: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 988 KiB/s wr, 103 op/s 2024-05-14T17:17:43.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 545212 3.71 MB/sec execute 595 sec latency 2200.312 ms 2024-05-14T17:17:43.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:43 smithi150 ceph-mon[36204]: pgmap v443: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 988 KiB/s wr, 103 op/s 2024-05-14T17:17:43.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:43 smithi099 ceph-mon[31851]: pgmap v443: 97 pgs: 97 active+clean; 163 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 988 KiB/s wr, 103 op/s 2024-05-14T17:17:44.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 545212 3.70 MB/sec execute 596 sec latency 3200.363 ms 2024-05-14T17:17:45.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:45 smithi193 ceph-mon[35962]: pgmap v444: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-14T17:17:45.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 547484 3.72 MB/sec execute 597 sec latency 3208.016 ms 2024-05-14T17:17:45.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:45 smithi150 ceph-mon[36204]: pgmap v444: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-14T17:17:45.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:45 smithi099 ceph-mon[31851]: pgmap v444: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-14T17:17:46.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 548762 3.72 MB/sec execute 598 sec latency 195.839 ms 2024-05-14T17:17:47.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:47 smithi193 ceph-mon[35962]: pgmap v445: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 108 op/s 2024-05-14T17:17:47.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 548762 3.71 MB/sec execute 599 sec latency 1195.945 ms 2024-05-14T17:17:47.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:47 smithi150 ceph-mon[36204]: pgmap v445: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 108 op/s 2024-05-14T17:17:47.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:47 smithi099 ceph-mon[31851]: pgmap v445: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 108 op/s 2024-05-14T17:17:48.568 INFO:tasks.workunit.client.0.smithi099.stdout: 1 cleanup 600 sec 2024-05-14T17:17:49.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:49 smithi193 ceph-mon[35962]: pgmap v446: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 108 op/s 2024-05-14T17:17:49.565 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:49 smithi099 ceph-mon[31851]: pgmap v446: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 108 op/s 2024-05-14T17:17:49.567 INFO:tasks.workunit.client.0.smithi099.stdout: 1 cleanup 601 sec 2024-05-14T17:17:49.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:49 smithi150 ceph-mon[36204]: pgmap v446: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 108 op/s 2024-05-14T17:17:49.803 INFO:tasks.workunit.client.0.smithi099.stdout: 0 cleanup 601 sec 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Operation Count AvgLat MaxLat 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: ---------------------------------------- 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: NTCreateX 70812 1.856 26587.391 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Close 52040 0.002 0.109 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Rename 3001 72.286 20280.922 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Unlink 14284 9.644 29713.834 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Deltree 2 74.125 109.426 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Mkdir 1 0.006 0.006 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Qpathinfo 64163 0.897 148.139 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Qfileinfo 11254 0.001 1.213 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Qfsinfo 11764 0.164 8.180 2024-05-14T17:17:49.804 INFO:tasks.workunit.client.0.smithi099.stdout: Sfileinfo 5782 0.034 9.944 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: Find 24808 0.202 61.299 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: WriteX 35349 0.963 31292.425 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: ReadX 110955 0.014 30.170 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: LockX 230 0.815 1.962 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: UnlockX 230 0.736 1.827 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: Flush 4966 2.655 52.958 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stdout:Throughput 3.71463 MB/sec 1 clients 1 procs max_latency=31292.434 ms 2024-05-14T17:17:49.805 INFO:tasks.workunit.client.0.smithi099.stderr:+ dbench 10 2024-05-14T17:17:49.806 INFO:tasks.workunit.client.0.smithi099.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-14T17:17:49.806 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-05-14T17:17:49.806 INFO:tasks.workunit.client.0.smithi099.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-14T17:17:49.806 INFO:tasks.workunit.client.0.smithi099.stdout:9 of 10 processes prepared for launch 0 sec 2024-05-14T17:17:49.905 INFO:tasks.workunit.client.0.smithi099.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-14T17:17:49.906 INFO:tasks.workunit.client.0.smithi099.stdout:releasing clients 2024-05-14T17:17:50.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 357 128.11 MB/sec warmup 1 sec latency 56.463 ms 2024-05-14T17:17:51.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:51 smithi193 ceph-mon[35962]: pgmap v447: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-14T17:17:51.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:51 smithi150 ceph-mon[36204]: pgmap v447: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-14T17:17:51.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:51 smithi099 ceph-mon[31851]: pgmap v447: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-05-14T17:17:51.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 763 102.71 MB/sec warmup 2 sec latency 874.660 ms 2024-05-14T17:17:52.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 1236 78.21 MB/sec warmup 3 sec latency 1874.722 ms 2024-05-14T17:17:53.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:53 smithi193 ceph-mon[35962]: pgmap v448: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 989 KiB/s wr, 105 op/s 2024-05-14T17:17:53.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:53 smithi150 ceph-mon[36204]: pgmap v448: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 989 KiB/s wr, 105 op/s 2024-05-14T17:17:53.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:53 smithi099 ceph-mon[31851]: pgmap v448: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 989 KiB/s wr, 105 op/s 2024-05-14T17:17:53.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 2003 63.68 MB/sec warmup 4 sec latency 2874.773 ms 2024-05-14T17:17:54.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 2556 60.45 MB/sec warmup 5 sec latency 3592.247 ms 2024-05-14T17:17:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 -- 172.21.15.99:0/988019853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c102340 msgr2=0x7f9b8c10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 --2- 172.21.15.99:0/988019853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c102340 0x7f9b8c10e800 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009cd0 tx=0x7f9b8402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 -- 172.21.15.99:0/988019853 shutdown_connections 2024-05-14T17:17:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 --2- 172.21.15.99:0/988019853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c102340 0x7f9b8c10e800 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 --2- 172.21.15.99:0/988019853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b8c101980 0x7f9b8c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 --2- 172.21.15.99:0/988019853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b8c107940 0x7f9b8c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 -- 172.21.15.99:0/988019853 >> 172.21.15.99:0/988019853 conn(0x7f9b8c078ad0 msgr2=0x7f9b8c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 -- 172.21.15.99:0/988019853 shutdown_connections 2024-05-14T17:17:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.398+0000 7f9b934dc640 1 -- 172.21.15.99:0/988019853 wait complete. 2024-05-14T17:17:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 Processor -- start 2024-05-14T17:17:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 -- start start 2024-05-14T17:17:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b8c101980 0x7f9b8c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b8c102340 0x7f9b8c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 0x7f9b8c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c112d00 con 0x7f9b8c107940 2024-05-14T17:17:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c112b80 con 0x7f9b8c101980 2024-05-14T17:17:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b934dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c112e80 con 0x7f9b8c102340 2024-05-14T17:17:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 0x7f9b8c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 0x7f9b8c19d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42312/0 (socket says 172.21.15.99:42312) 2024-05-14T17:17:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 -- 172.21.15.99:0/3924574555 learned_addr learned my addr 172.21.15.99:0/3924574555 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b90a77640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b8c102340 0x7f9b8c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91278640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b8c101980 0x7f9b8c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 -- 172.21.15.99:0/3924574555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b8c102340 msgr2=0x7f9b8c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b8c102340 0x7f9b8c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 -- 172.21.15.99:0/3924574555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b8c101980 msgr2=0x7f9b8c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b8c101980 0x7f9b8c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 -- 172.21.15.99:0/3924574555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c19e120 con 0x7f9b8c107940 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.399+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 0x7f9b8c19d9f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9b84033040 tx=0x7f9b84009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.400+0000 7f9b827fc640 1 -- 172.21.15.99:0/3924574555 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84004040 con 0x7f9b8c107940 2024-05-14T17:17:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.400+0000 7f9b827fc640 1 -- 172.21.15.99:0/3924574555 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b84004690 con 0x7f9b8c107940 2024-05-14T17:17:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.400+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c19e3b0 con 0x7f9b8c107940 2024-05-14T17:17:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.400+0000 7f9b827fc640 1 -- 172.21.15.99:0/3924574555 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84030d90 con 0x7f9b8c107940 2024-05-14T17:17:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.400+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c1a5ca0 con 0x7f9b8c107940 2024-05-14T17:17:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.402+0000 7f9b827fc640 1 -- 172.21.15.99:0/3924574555 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b840041e0 con 0x7f9b8c107940 2024-05-14T17:17:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 Processor -- start 2024-05-14T17:17:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 -- start start 2024-05-14T17:17:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b827fc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9b780b2a90 0x7f9b780b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a63ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5638839c9c30 con 0x563883a568f0 2024-05-14T17:17:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5638839b97d0 con 0x563883a61ad0 2024-05-14T17:17:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b961f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638839bd810 con 0x563883a57210 2024-05-14T17:17:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b90a77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91278640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9b780b2a90 0x7f9b780b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b90a77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a56cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42314/0 (socket says 172.21.15.99:42314) 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91a79640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a63ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b90a77640 1 -- 172.21.15.99:0/1031841190 learned_addr learned my addr 172.21.15.99:0/1031841190 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b90a77640 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b827fc640 1 -- 172.21.15.99:0/3924574555 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f9b8402e030 con 0x7f9b8c107940 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91a79640 1 -- 172.21.15.99:0/1031841190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 msgr2=0x563883a61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91a79640 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a61590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91a79640 1 -- 172.21.15.99:0/1031841190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 msgr2=0x563883a56cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91a79640 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a56cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b91a79640 1 -- 172.21.15.99:0/1031841190 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638836ce210 con 0x563883a61ad0 2024-05-14T17:17:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.403+0000 7f9b90a77640 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a56cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:17:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b91278640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9b780b2a90 0x7f9b780b4f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009b50 tx=0x7f9b88005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b91a79640 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a63ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b84100c10 tx=0x7f9b84005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ffddf7968c15cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b777fe640 1 -- 172.21.15.99:0/1031841190 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b840caf30 con 0x563883a61ad0 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b777fe640 1 -- 172.21.15.99:0/1031841190 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b840044d0 con 0x563883a61ad0 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b777fe640 1 -- 172.21.15.99:0/1031841190 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8404a1f0 con 0x563883a61ad0 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b961f5740 1 -- 172.21.15.99:0/1031841190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 msgr2=0x563883a63ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.404+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a63ec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b84100c10 tx=0x7f9b84005f80 comp rx=0 tx=0).stop 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- 172.21.15.99:0/1031841190 shutdown_connections 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a63ec0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1031841190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a56cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- 172.21.15.99:0/1031841190 >> 172.21.15.99:0/1031841190 conn(0x563883a52e00 msgr2=0x563883a531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- 172.21.15.99:0/1031841190 shutdown_connections 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- 172.21.15.99:0/1031841190 wait complete. 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- start start 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 Processor -- start 2024-05-14T17:17:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a568f0 0x563883a61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 0x563883a5e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a5ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56388369f790 con 0x563883a57210 2024-05-14T17:17:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563883975890 con 0x563883a61ad0 2024-05-14T17:17:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563883a644c0 con 0x563883a568f0 2024-05-14T17:17:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b961f5740 10 client.0 _ll_register_callbacks cb 0x563883978d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b90a77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 0x563883a5e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.405+0000 7f9b90a77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 0x563883a5e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42320/0 (socket says 172.21.15.99:42320) 2024-05-14T17:17:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 -- 172.21.15.99:0/3247849875 learned_addr learned my addr 172.21.15.99:0/3247849875 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b91278640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a568f0 0x563883a61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a5ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 -- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a568f0 msgr2=0x563883a61430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a568f0 0x563883a61430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 -- 172.21.15.99:0/3247849875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 msgr2=0x563883a5ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a5ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 -- 172.21.15.99:0/3247849875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388379a620 con 0x563883a57210 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b90a77640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 0x563883a5e720 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c013aa0 tx=0x7f9b7c013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b351c6bf2e59f556 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3247849875 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c010e50 con 0x563883a57210 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3247849875 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f9b7c0047a0 con 0x563883a57210 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638839cbb80 con 0x563883a57210 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3247849875 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c026050 con 0x563883a57210 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b961f5740 10 client.16992 fetch_fsmap learned FSMap version 44 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563883a6ff00 con 0x563883a57210 2024-05-14T17:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.406+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3247849875 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5638839cbb80 con 0x563883a57210 2024-05-14T17:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3247849875 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f9b7c01e3c0 con 0x563883a57210 2024-05-14T17:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b961f5740 10 client.16992 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b961f5740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b961f5740 20 client.16992 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883a713e0 0x563883a737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b961f5740 4 client.16992 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:17:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638839c5360 con 0x563883a713e0 2024-05-14T17:17:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b91278640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883a713e0 0x563883a737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b74ff9640 10 client.16992 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.407+0000 7f9b91278640 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883a713e0 0x563883a737d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=289ad5df7833de0e server_cookie=aa215ced4034df35 in_seq=0 out_seq=0 2024-05-14T17:17:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.412+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3247849875 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638839c5360 con 0x563883a713e0 2024-05-14T17:17:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.412+0000 7f9b74ff9640 10 client.16992 handle_command_reply: tid=0 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.412+0000 7f9b961f5740 1 client.16992 shutdown 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 msgr2=0x563883a5e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 0x563883a5e720 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c013aa0 tx=0x7f9b7c013f60 comp rx=0 tx=0).stop 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 shutdown_connections 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883a713e0 0x563883a737d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a61ad0 0x563883a5ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b74ff9640 0 client.16992 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a57210 0x563883a5e720 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3247849875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a568f0 0x563883a61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 >> 172.21.15.99:0/3247849875 conn(0x563883a52e00 msgr2=0x563883a52100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 shutdown_connections 2024-05-14T17:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 1 -- 172.21.15.99:0/3247849875 wait complete. 2024-05-14T17:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.413+0000 7f9b961f5740 20 client.16992 trim_cache size 0 max 16384 2024-05-14T17:17:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.428+0000 7f9b961f5740 1 Processor -- start 2024-05-14T17:17:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.428+0000 7f9b961f5740 1 -- start start 2024-05-14T17:17:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.428+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a60f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.428+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a57210 0x563883a589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a61ad0 0x563883a790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b961f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1b0e30 con 0x563883a568f0 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b961f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ae6d0 con 0x563883a57210 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b961f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ac540 con 0x563883a61ad0 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a60f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a60f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42324/0 (socket says 172.21.15.99:42324) 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 -- 172.21.15.99:0/1792950021 learned_addr learned my addr 172.21.15.99:0/1792950021 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b90a77640 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a57210 0x563883a589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91a79640 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a61ad0 0x563883a790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 -- 172.21.15.99:0/1792950021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a61ad0 msgr2=0x563883a790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a61ad0 0x563883a790f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 -- 172.21.15.99:0/1792950021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a57210 msgr2=0x563883a589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a57210 0x563883a589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 -- 172.21.15.99:0/1792950021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563883a52c30 con 0x563883a568f0 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b90a77640 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a57210 0x563883a589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:17:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b91278640 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a60f60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9b880118d0 tx=0x7f9b88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5388491d62d0974f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b76ffd640 1 -- 172.21.15.99:0/1792950021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8802d070 con 0x563883a568f0 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b76ffd640 1 -- 172.21.15.99:0/1792950021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b88020190 con 0x563883a568f0 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.429+0000 7f9b76ffd640 1 -- 172.21.15.99:0/1792950021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800e950 con 0x563883a568f0 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 -- 172.21.15.99:0/1792950021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 msgr2=0x563883a60f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a60f60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9b880118d0 tx=0x7f9b88014000 comp rx=0 tx=0).stop 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 -- 172.21.15.99:0/1792950021 shutdown_connections 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a61ad0 0x563883a790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a57210 0x563883a589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 --2- 172.21.15.99:0/1792950021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a568f0 0x563883a60f60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 -- 172.21.15.99:0/1792950021 >> 172.21.15.99:0/1792950021 conn(0x563883a52e00 msgr2=0x563883a582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 -- 172.21.15.99:0/1792950021 shutdown_connections 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 -- 172.21.15.99:0/1792950021 wait complete. 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 -- start start 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.430+0000 7f9b961f5740 1 Processor -- start 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a568f0 0x563883a69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 0x563883a65700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1acb90 con 0x563883a61ad0 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ae000 con 0x563883a568f0 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c1ad260 con 0x563883a57210 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 10 client.0 _ll_register_callbacks cb 0x7f9b7c0136a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 0x563883a65700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 0x563883a65700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42326/0 (socket says 172.21.15.99:42326) 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91278640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a568f0 0x563883a69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 -- 172.21.15.99:0/3835590369 learned_addr learned my addr 172.21.15.99:0/3835590369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b90a77640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 -- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 msgr2=0x563883a650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 -- 172.21.15.99:0/3835590369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a568f0 msgr2=0x563883a69d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:17:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a568f0 0x563883a69d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 -- 172.21.15.99:0/3835590369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563883a65f30 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91278640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a568f0 0x563883a69d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b91a79640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 0x563883a65700 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9b840c7aa0 tx=0x7f9b84031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422ea38aff2cb1aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3835590369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b840d85e0 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3835590369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f9b840d8210 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.431+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b88020190 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3835590369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9b840e4050 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 10 client.16998 fetch_fsmap learned FSMap version 44 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563883adbde0 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3835590369 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b840e8080 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3835590369 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f9b84103030 con 0x563883a61ad0 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 10 client.16998 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:17:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 20 client.16998 populate_metadata read hostname 'smithi099' 2024-05-14T17:17:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883adc370 0x563883ade760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 4 client.16998 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:17:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56388393f5f0 con 0x563883adc370 2024-05-14T17:17:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.432+0000 7f9b91278640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883adc370 0x563883ade760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.433+0000 7f9b91278640 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883adc370 0x563883ade760 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc53cc0551c6c0f server_cookie=718ddb6b6a4b2c4c in_seq=0 out_seq=0 2024-05-14T17:17:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.433+0000 7f9b74ff9640 10 client.16998 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:55.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:55 smithi150 ceph-mon[36204]: pgmap v449: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b74ff9640 1 -- 172.21.15.99:0/3835590369 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56388393f5f0 con 0x563883adc370 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b74ff9640 10 client.16998 handle_command_reply: tid=0 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 client.16998 shutdown 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 msgr2=0x563883a65700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 0x563883a65700 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9b840c7aa0 tx=0x7f9b84031df0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 shutdown_connections 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563883adc370 0x563883ade760 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563883a61ad0 0x563883a65700 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563883a57210 0x563883a650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b74ff9640 0 client.16998 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 --2- 172.21.15.99:0/3835590369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563883a568f0 0x563883a69d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 >> 172.21.15.99:0/3835590369 conn(0x563883a52e00 msgr2=0x563883a58b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 shutdown_connections 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.757+0000 7f9b961f5740 1 -- 172.21.15.99:0/3835590369 wait complete. 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b961f5740 20 client.16998 trim_cache size 0 max 16384 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:17:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:17:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9b780b2a90 msgr2=0x7f9b780b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9b780b2a90 0x7f9b780b4f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009b50 tx=0x7f9b88005f90 comp rx=0 tx=0).stop 2024-05-14T17:17:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 msgr2=0x7f9b8c19d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 0x7f9b8c19d9f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9b84033040 tx=0x7f9b84009360 comp rx=0 tx=0).stop 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 shutdown_connections 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9b780b2a90 0x7f9b780b4f50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b8c107940 0x7f9b8c19d9f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b8c102340 0x7f9b8c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 --2- 172.21.15.99:0/3924574555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b8c101980 0x7f9b8c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 >> 172.21.15.99:0/3924574555 conn(0x7f9b8c078ad0 msgr2=0x7f9b8c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 shutdown_connections 2024-05-14T17:17:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:17:55.758+0000 7f9b934dc640 1 -- 172.21.15.99:0/3924574555 wait complete. 2024-05-14T17:17:55.763 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:55 smithi099 ceph-mon[31851]: pgmap v449: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-14T17:17:55.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:55 smithi193 ceph-mon[35962]: pgmap v449: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-14T17:17:55.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 2952 54.78 MB/sec warmup 6 sec latency 179.010 ms 2024-05-14T17:17:56.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:56 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:56.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:56 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:56.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:56 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:56.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3495 54.40 MB/sec warmup 7 sec latency 101.162 ms 2024-05-14T17:17:57.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:57 smithi150 ceph-mon[36204]: pgmap v450: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-05-14T17:17:57.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:57 smithi099 ceph-mon[31851]: pgmap v450: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-05-14T17:17:57.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:57 smithi193 ceph-mon[35962]: pgmap v450: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-05-14T17:17:57.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3685 48.33 MB/sec warmup 8 sec latency 74.858 ms 2024-05-14T17:17:58.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 3875 43.00 MB/sec warmup 9 sec latency 603.528 ms 2024-05-14T17:17:59.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:17:59 smithi150 ceph-mon[36204]: pgmap v451: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-05-14T17:17:59.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:17:59 smithi099 ceph-mon[31851]: pgmap v451: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-05-14T17:17:59.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:17:59 smithi193 ceph-mon[35962]: pgmap v451: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-05-14T17:17:59.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 4105 39.08 MB/sec warmup 10 sec latency 1269.793 ms 2024-05-14T17:18:00.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 4374 36.68 MB/sec warmup 11 sec latency 278.723 ms 2024-05-14T17:18:01.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:01 smithi150 ceph-mon[36204]: pgmap v452: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 415 op/s 2024-05-14T17:18:01.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:01 smithi099 ceph-mon[31851]: pgmap v452: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 415 op/s 2024-05-14T17:18:01.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:01 smithi193 ceph-mon[35962]: pgmap v452: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 415 op/s 2024-05-14T17:18:01.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 4756 36.27 MB/sec warmup 12 sec latency 429.365 ms 2024-05-14T17:18:02.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:02 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:18:02.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:02 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:18:02.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:02 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:18:02.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 5423 34.32 MB/sec warmup 13 sec latency 58.387 ms 2024-05-14T17:18:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:03 smithi150 ceph-mon[36204]: pgmap v453: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-05-14T17:18:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:03 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:18:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:03 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:18:03.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:03 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:18:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:03 smithi099 ceph-mon[31851]: pgmap v453: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-05-14T17:18:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:03 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:18:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:03 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:18:03.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:03 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:18:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:03 smithi193 ceph-mon[35962]: pgmap v453: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-05-14T17:18:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:03 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:18:03.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:03 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:18:03.793 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:03 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:18:03.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 5883 33.66 MB/sec warmup 14 sec latency 44.729 ms 2024-05-14T17:18:04.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 6496 34.84 MB/sec warmup 15 sec latency 49.848 ms 2024-05-14T17:18:05.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:05 smithi150 ceph-mon[36204]: pgmap v454: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 676 op/s 2024-05-14T17:18:05.763 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:05 smithi099 ceph-mon[31851]: pgmap v454: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 676 op/s 2024-05-14T17:18:05.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:05 smithi193 ceph-mon[35962]: pgmap v454: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 676 op/s 2024-05-14T17:18:05.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 6869 34.78 MB/sec warmup 16 sec latency 607.057 ms 2024-05-14T17:18:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:06 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:06.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:06 smithi150 ceph-mon[36204]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:18:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:06 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:06.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:06 smithi099 ceph-mon[31851]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:18:06.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:06 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:06.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:06 smithi193 ceph-mon[35962]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:18:06.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7036 33.15 MB/sec warmup 17 sec latency 1607.084 ms 2024-05-14T17:18:07.705 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:07 smithi150 ceph-mon[36204]: pgmap v456: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 725 op/s 2024-05-14T17:18:07.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:07 smithi099 ceph-mon[31851]: pgmap v456: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 725 op/s 2024-05-14T17:18:07.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:07 smithi193 ceph-mon[35962]: pgmap v456: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 725 op/s 2024-05-14T17:18:07.907 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7223 31.42 MB/sec warmup 18 sec latency 2607.191 ms 2024-05-14T17:18:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 -- 172.21.15.99:0/3022289503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc101980 msgr2=0x7f88bc101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 --2- 172.21.15.99:0/3022289503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc101980 0x7f88bc101e00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009930 tx=0x7f88a402a420 comp rx=0 tx=0).stop 2024-05-14T17:18:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 -- 172.21.15.99:0/3022289503 shutdown_connections 2024-05-14T17:18:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 --2- 172.21.15.99:0/3022289503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88bc102340 0x7f88bc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 --2- 172.21.15.99:0/3022289503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc101980 0x7f88bc101e00 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 --2- 172.21.15.99:0/3022289503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88bc107940 0x7f88bc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 -- 172.21.15.99:0/3022289503 >> 172.21.15.99:0/3022289503 conn(0x7f88bc078ad0 msgr2=0x7f88bc078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 -- 172.21.15.99:0/3022289503 shutdown_connections 2024-05-14T17:18:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.852+0000 7f88c1045640 1 -- 172.21.15.99:0/3022289503 wait complete. 2024-05-14T17:18:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.853+0000 7f88c1045640 1 Processor -- start 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.853+0000 7f88c1045640 1 -- start start 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88bc101980 0x7f88bc19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 0x7f88bc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88bc107940 0x7f88bc1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc111c50 con 0x7f88bc102340 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc111ad0 con 0x7f88bc107940 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc111dd0 con 0x7f88bc101980 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 0x7f88bc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 0x7f88bc19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46648/0 (socket says 172.21.15.99:46648) 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 -- 172.21.15.99:0/2512064526 learned_addr learned my addr 172.21.15.99:0/2512064526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88bc101980 0x7f88bc19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88bb59e640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88bc107940 0x7f88bc1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 -- 172.21.15.99:0/2512064526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88bc101980 msgr2=0x7f88bc19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88bc101980 0x7f88bc19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 -- 172.21.15.99:0/2512064526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88bc107940 msgr2=0x7f88bc1a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88bc107940 0x7f88bc1a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 -- 172.21.15.99:0/2512064526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc1a56b0 con 0x7f88bc102340 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b259c640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 0x7f88bc19a630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005ab0 tx=0x7f88a4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b8d99640 1 -- 172.21.15.99:0/2512064526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a403f070 con 0x7f88bc102340 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b8d99640 1 -- 172.21.15.99:0/2512064526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a4031b50 con 0x7f88bc102340 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc1a5880 con 0x7f88bc102340 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88b8d99640 1 -- 172.21.15.99:0/2512064526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4035030 con 0x7f88bc102340 2024-05-14T17:18:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.854+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a5ce0 con 0x7f88bc102340 2024-05-14T17:18:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.857+0000 7f88c39fd740 1 Processor -- start 2024-05-14T17:18:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.857+0000 7f88c39fd740 1 -- start start 2024-05-14T17:18:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88b8d99640 1 -- 172.21.15.99:0/2512064526 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f88a4042070 con 0x7f88bc102340 2024-05-14T17:18:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a07cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a12590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a12ad0 0x55bd62a14ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88c39fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bd6297ac30 con 0x55bd62a08210 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88c39fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd6296a7d0 con 0x55bd62a12ad0 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88c39fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd6296e810 con 0x55bd62a078f0 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a12590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bb59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a07cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a12590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46656/0 (socket says 172.21.15.99:46656) 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bad9d640 1 -- 172.21.15.99:0/562251147 learned_addr learned my addr 172.21.15.99:0/562251147 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88b259c640 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a12ad0 0x55bd62a14ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bb59e640 1 -- 172.21.15.99:0/562251147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a12ad0 msgr2=0x55bd62a14ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bb59e640 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a12ad0 0x55bd62a14ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bb59e640 1 -- 172.21.15.99:0/562251147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 msgr2=0x55bd62a12590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bb59e640 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a12590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bb59e640 1 -- 172.21.15.99:0/562251147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6267f210 con 0x55bd62a078f0 2024-05-14T17:18:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.858+0000 7f88bad9d640 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a12590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88bb59e640 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a07cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009f00 tx=0x7f88b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978197271bd10d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88b8d99640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88a80b2a90 0x7f88a80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88b159a640 1 -- 172.21.15.99:0/562251147 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004d20 con 0x55bd62a078f0 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88a80b2a90 0x7f88a80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88b159a640 1 -- 172.21.15.99:0/562251147 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b4004030 con 0x55bd62a078f0 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88b159a640 1 -- 172.21.15.99:0/562251147 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4005260 con 0x55bd62a078f0 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.859+0000 7f88b8d99640 1 -- 172.21.15.99:0/2512064526 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f88a4102c20 con 0x7f88bc102340 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- 172.21.15.99:0/562251147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 msgr2=0x55bd62a07cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88a80b2a90 0x7f88a80b4f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55bd62a08e90 tx=0x7f88ac00a2d0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a07cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009f00 tx=0x7f88b4011810 comp rx=0 tx=0).stop 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- 172.21.15.99:0/562251147 shutdown_connections 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a12ad0 0x55bd62a14ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a12590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- 172.21.15.99:0/562251147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a07cd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- 172.21.15.99:0/562251147 >> 172.21.15.99:0/562251147 conn(0x55bd62a03e00 msgr2=0x55bd62a041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- 172.21.15.99:0/562251147 shutdown_connections 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- 172.21.15.99:0/562251147 wait complete. 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- start start 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 Processor -- start 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a12440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a0f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 0x55bd62a0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bd62650790 con 0x55bd62a12ad0 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd62926890 con 0x55bd62a08210 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.860+0000 7f88c39fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd62a157a0 con 0x55bd62a078f0 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88c39fd740 10 client.0 _ll_register_callbacks cb 0x55bd62929d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 0x55bd62a0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a12440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 0x55bd62a0fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46672/0 (socket says 172.21.15.99:46672) 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 -- 172.21.15.99:0/1228900417 learned_addr learned my addr 172.21.15.99:0/1228900417 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88b259c640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a0f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 -- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 msgr2=0x55bd62a12440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a12440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 -- 172.21.15.99:0/1228900417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 msgr2=0x55bd62a0f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a0f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 -- 172.21.15.99:0/1228900417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6274b620 con 0x55bd62a12ad0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bad9d640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a12440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88bb59e640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 0x55bd62a0fc90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003b40 tx=0x7f88b4003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23c049b9bd8afc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88a27fc640 1 -- 172.21.15.99:0/1228900417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4027020 con 0x55bd62a12ad0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88a27fc640 1 -- 172.21.15.99:0/1228900417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f88b40043f0 con 0x55bd62a12ad0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b4004030 con 0x55bd62a12ad0 2024-05-14T17:18:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88a27fc640 1 -- 172.21.15.99:0/1228900417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f88b4032050 con 0x55bd62a12ad0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88c39fd740 10 client.17010 fetch_fsmap learned FSMap version 45 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.861+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bd62a211c0 con 0x55bd62a12ad0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88a27fc640 1 -- 172.21.15.99:0/1228900417 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b4004030 con 0x55bd62a12ad0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88a27fc640 1 -- 172.21.15.99:0/1228900417 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f88b4018070 con 0x55bd62a12ad0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88c39fd740 10 client.17010 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88c39fd740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88c39fd740 20 client.17010 populate_metadata read hostname 'smithi099' 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88c39fd740 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a225c0 0x55bd62a249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88c39fd740 4 client.17010 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd6297cb80 con 0x55bd62a225c0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88bad9d640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a225c0 0x55bd62a249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.862+0000 7f88bad9d640 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a225c0 0x55bd62a249b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b22ea12b5e20ee server_cookie=c21c3ef190631de8 in_seq=0 out_seq=0 2024-05-14T17:18:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.863+0000 7f88a27fc640 10 client.17010 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.863+0000 7f88a27fc640 1 -- 172.21.15.99:0/1228900417 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd6297cb80 con 0x55bd62a225c0 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.863+0000 7f88a27fc640 10 client.17010 handle_command_reply: tid=0 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.863+0000 7f88c39fd740 1 client.17010 shutdown 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 msgr2=0x55bd62a0fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 0x55bd62a0fc90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003b40 tx=0x7f88b4003b70 comp rx=0 tx=0).stop 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 shutdown_connections 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a225c0 0x55bd62a249b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a12ad0 0x55bd62a0fc90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a0f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 --2- 172.21.15.99:0/1228900417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a078f0 0x55bd62a12440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 >> 172.21.15.99:0/1228900417 conn(0x55bd62a03e00 msgr2=0x55bd62a03100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88a27fc640 0 client.17010 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 shutdown_connections 2024-05-14T17:18:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 1 -- 172.21.15.99:0/1228900417 wait complete. 2024-05-14T17:18:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.864+0000 7f88c39fd740 20 client.17010 trim_cache size 0 max 16384 2024-05-14T17:18:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 Processor -- start 2024-05-14T17:18:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 -- start start 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 0x55bd62a0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a11f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1b0f90 con 0x55bd62a078f0 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1ae800 con 0x55bd62a08210 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88c39fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1ac670 con 0x55bd62a12ad0 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88bad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 0x55bd62a0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88bad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 0x55bd62a0fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46680/0 (socket says 172.21.15.99:46680) 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88bad9d640 1 -- 172.21.15.99:0/2622586806 learned_addr learned my addr 172.21.15.99:0/2622586806 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88b259c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.881+0000 7f88bb59e640 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a11f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88bad9d640 1 -- 172.21.15.99:0/2622586806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 msgr2=0x55bd62a11f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a11f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88bad9d640 1 -- 172.21.15.99:0/2622586806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 msgr2=0x55bd62a03340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a03340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88bad9d640 1 -- 172.21.15.99:0/2622586806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd62a03c30 con 0x55bd62a078f0 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88b259c640 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a03340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 0x55bd62a0fc80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f88ac005c50 tx=0x7f88ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5112069d28999a23 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88b0d99640 1 -- 172.21.15.99:0/2622586806 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac031030 con 0x55bd62a078f0 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88b0d99640 1 -- 172.21.15.99:0/2622586806 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac0035f0 con 0x55bd62a078f0 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.882+0000 7f88b0d99640 1 -- 172.21.15.99:0/2622586806 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac02c0f0 con 0x55bd62a078f0 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 -- 172.21.15.99:0/2622586806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 msgr2=0x55bd62a0fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 0x55bd62a0fc80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f88ac005c50 tx=0x7f88ac019000 comp rx=0 tx=0).stop 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 -- 172.21.15.99:0/2622586806 shutdown_connections 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a11f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a08210 0x55bd62a03340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2622586806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a078f0 0x55bd62a0fc80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 -- 172.21.15.99:0/2622586806 >> 172.21.15.99:0/2622586806 conn(0x55bd62a03e00 msgr2=0x55bd62a09b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 -- 172.21.15.99:0/2622586806 shutdown_connections 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 -- 172.21.15.99:0/2622586806 wait complete. 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 -- start start 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 Processor -- start 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a078f0 0x55bd62a1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.883+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a16030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a16570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1accc0 con 0x55bd62a08210 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1ae130 con 0x55bd62a078f0 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1ad390 con 0x55bd62a12ad0 2024-05-14T17:18:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 10 client.0 _ll_register_callbacks cb 0x7f88b4004ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a16030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a16030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46688/0 (socket says 172.21.15.99:46688) 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 -- 172.21.15.99:0/2095986730 learned_addr learned my addr 172.21.15.99:0/2095986730 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88bb59e640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a16570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a078f0 0x55bd62a1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 -- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 msgr2=0x55bd62a16570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a16570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 -- 172.21.15.99:0/2095986730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a078f0 msgr2=0x55bd62a1ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a078f0 0x55bd62a1ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 -- 172.21.15.99:0/2095986730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd62a16ab0 con 0x55bd62a08210 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a078f0 0x55bd62a1ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88b259c640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a16030 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f88a40c6000 tx=0x7f88a402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2d3f1d6c6107a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:18:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88a27fc640 1 -- 172.21.15.99:0/2095986730 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4049040 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88a27fc640 1 -- 172.21.15.99:0/2095986730 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f88a40c6210 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac0035f0 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88a27fc640 1 -- 172.21.15.99:0/2095986730 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f88a40c9f80 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 10 client.17016 fetch_fsmap learned FSMap version 45 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bd62a8ca90 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.884+0000 7f88a27fc640 1 -- 172.21.15.99:0/2095986730 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac0035f0 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88a27fc640 1 -- 172.21.15.99:0/2095986730 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f88a4108070 con 0x55bd62a08210 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88c39fd740 10 client.17016 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88c39fd740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88c39fd740 20 client.17016 populate_metadata read hostname 'smithi099' 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a8d290 0x55bd62a8f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88c39fd740 4 client.17016 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd62976360 con 0x55bd62a8d290 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a8d290 0x55bd62a8f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88a27fc640 10 client.17016 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:08.885+0000 7f88bad9d640 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a8d290 0x55bd62a8f6b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda54af6c72794a7 server_cookie=92036ad1f4aa005 in_seq=0 out_seq=0 2024-05-14T17:18:08.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7510 30.23 MB/sec warmup 19 sec latency 3607.232 ms 2024-05-14T17:18:09.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:09 smithi150 ceph-mon[36204]: pgmap v457: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 725 op/s 2024-05-14T17:18:09.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:09 smithi099 ceph-mon[31851]: pgmap v457: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 725 op/s 2024-05-14T17:18:09.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:09 smithi193 ceph-mon[35962]: pgmap v457: 97 pgs: 97 active+clean; 196 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 725 op/s 2024-05-14T17:18:09.906 INFO:tasks.workunit.client.0.smithi099.stdout: 10 7910 30.22 MB/sec warmup 20 sec latency 4607.299 ms 2024-05-14T17:18:10.907 INFO:tasks.workunit.client.0.smithi099.stdout: 10 8520 29.54 MB/sec warmup 21 sec latency 5607.342 ms 2024-05-14T17:18:11.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:11 smithi150 ceph-mon[36204]: pgmap v458: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-05-14T17:18:11.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:11 smithi099 ceph-mon[31851]: pgmap v458: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-05-14T17:18:11.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:11 smithi193 ceph-mon[35962]: pgmap v458: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-05-14T17:18:11.907 INFO:tasks.workunit.client.0.smithi099.stdout: 10 9147 30.84 MB/sec warmup 22 sec latency 6607.410 ms 2024-05-14T17:18:12.907 INFO:tasks.workunit.client.0.smithi099.stdout: 10 9391 29.94 MB/sec warmup 23 sec latency 7607.451 ms 2024-05-14T17:18:13.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:13 smithi150 ceph-mon[36204]: pgmap v459: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-05-14T17:18:13.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:13 smithi099 ceph-mon[31851]: pgmap v459: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-05-14T17:18:13.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:13 smithi193 ceph-mon[35962]: pgmap v459: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 480 op/s 2024-05-14T17:18:13.907 INFO:tasks.workunit.client.0.smithi099.stdout: 10 9562 28.72 MB/sec warmup 24 sec latency 8607.520 ms 2024-05-14T17:18:14.907 INFO:tasks.workunit.client.0.smithi099.stdout: 10 9905 28.05 MB/sec warmup 25 sec latency 9607.561 ms 2024-05-14T17:18:15.186 INFO:tasks.workunit.client.0.smithi099.stdout:[5882] write failed on handle 11123 (Resource temporarily unavailable) 2024-05-14T17:18:15.186 INFO:tasks.workunit.client.0.smithi099.stdout:Child failed with status 1 2024-05-14T17:18:15.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:18:15.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:15 smithi193 ceph-mon[35962]: pgmap v460: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:15.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:15 smithi193 ceph-mon[35962]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:18:15.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:15 smithi150 ceph-mon[36204]: pgmap v460: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:15.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:15 smithi150 ceph-mon[36204]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:18:15.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:15 smithi099 ceph-mon[31851]: pgmap v460: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:15.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:15 smithi099 ceph-mon[31851]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:18:17.337 INFO:tasks.quiescer.fs.[cephfs]:Total duration (36.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T17:18:17.337 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.4 2024-05-14T17:18:17.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:18:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:17.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:17 smithi150 ceph-mon[36204]: pgmap v462: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:17.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:17 smithi099 ceph-mon[31851]: pgmap v462: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:17.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:17 smithi193 ceph-mon[35962]: pgmap v462: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:19.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 -- 172.21.15.99:0/1112951183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463806bbe0 msgr2=0x7f46381103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:19.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 --2- 172.21.15.99:0/1112951183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463806bbe0 0x7f46381103d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f463000b0a0 tx=0x7f463000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 -- 172.21.15.99:0/1112951183 shutdown_connections 2024-05-14T17:18:19.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 --2- 172.21.15.99:0/1112951183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f463806bbe0 0x7f46381103d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 --2- 172.21.15.99:0/1112951183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46381095b0 0x7f4638109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 --2- 172.21.15.99:0/1112951183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f463806b220 0x7f463806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 -- 172.21.15.99:0/1112951183 >> 172.21.15.99:0/1112951183 conn(0x7f46380793f0 msgr2=0x7f4638079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:19.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 -- 172.21.15.99:0/1112951183 shutdown_connections 2024-05-14T17:18:19.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.290+0000 7f463ee68700 1 -- 172.21.15.99:0/1112951183 wait complete. 2024-05-14T17:18:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 Processor -- start 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 -- start start 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f463806b220 0x7f4638102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 0x7f4638102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46381095b0 0x7f4638102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4638113710 con 0x7f463806bbe0 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4638113590 con 0x7f46381095b0 2024-05-14T17:18:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463ee68700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4638113890 con 0x7f463806b220 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f4637fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 0x7f4638102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f4637fff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 0x7f4638102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43960/0 (socket says 172.21.15.99:43960) 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f4637fff700 1 -- 172.21.15.99:0/3520979570 learned_addr learned my addr 172.21.15.99:0/3520979570 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463cc04700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f463806b220 0x7f4638102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.291+0000 7f463d405700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46381095b0 0x7f4638102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4637fff700 1 -- 172.21.15.99:0/3520979570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f463806b220 msgr2=0x7f4638102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:19.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4637fff700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f463806b220 0x7f4638102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4637fff700 1 -- 172.21.15.99:0/3520979570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46381095b0 msgr2=0x7f4638102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:19.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4637fff700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46381095b0 0x7f4638102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4637fff700 1 -- 172.21.15.99:0/3520979570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46381a8110 con 0x7f463806bbe0 2024-05-14T17:18:19.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4637fff700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 0x7f4638102a50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f462400d950 tx=0x7f462400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:19.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624016070 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4624009df0 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400f3c0 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46381a8400 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.292+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381a88f0 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.293+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463806a230 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.294+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4624019070 con 0x7f463806bbe0 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.295+0000 7f4635ffb700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f46280b2a70 0x7f46280b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:19.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.295+0000 7f463cc04700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f46280b2a70 0x7f46280b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.295+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f46240e21c0 con 0x7f463806bbe0 2024-05-14T17:18:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.296+0000 7f463cc04700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f46280b2a70 0x7f46280b4f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f462c009be0 tx=0x7f462c009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.300+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46240a9ba0 con 0x7f463806bbe0 2024-05-14T17:18:19.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.556+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f463802e720 con 0x7f463806bbe0 2024-05-14T17:18:19.555 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:19 smithi099 ceph-mon[31851]: pgmap v463: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:19.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.556+0000 7f4635ffb700 1 -- 172.21.15.99:0/3520979570 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11876 (secure 0 0 0) 0x7f46240a94a0 con 0x7f463806bbe0 2024-05-14T17:18:19.556 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:18:19.556 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:18:05.015993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:18:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.559+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f46280b2a70 msgr2=0x7f46280b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.559+0000 7f463ee68700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f46280b2a70 0x7f46280b4f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f462c009be0 tx=0x7f462c009260 comp rx=0 tx=0).stop 2024-05-14T17:18:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.559+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 msgr2=0x7f4638102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.559+0000 7f463ee68700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 0x7f4638102a50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f462400d950 tx=0x7f462400dc60 comp rx=0 tx=0).stop 2024-05-14T17:18:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 shutdown_connections 2024-05-14T17:18:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f46280b2a70 0x7f46280b4f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46381095b0 0x7f4638102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f463806b220 0x7f4638102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 --2- 172.21.15.99:0/3520979570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f463806bbe0 0x7f4638102a50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 >> 172.21.15.99:0/3520979570 conn(0x7f46380793f0 msgr2=0x7f4638101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 shutdown_connections 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:19.560+0000 7f463ee68700 1 -- 172.21.15.99:0/3520979570 wait complete. 2024-05-14T17:18:19.560 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 45 2024-05-14T17:18:19.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:19 smithi150 ceph-mon[36204]: pgmap v463: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:19.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:19 smithi193 ceph-mon[35962]: pgmap v463: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-05-14T17:18:20.106 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-14T17:18:20.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-14T17:18:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88a27fc640 1 -- 172.21.15.99:0/2095986730 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd62976360 con 0x55bd62a8d290 2024-05-14T17:18:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88a27fc640 10 client.17016 handle_command_reply: tid=0 2024-05-14T17:18:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 client.17016 shutdown 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 msgr2=0x55bd62a16030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a16030 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f88a40c6000 tx=0x7f88a402a420 comp rx=0 tx=0).stop 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 shutdown_connections 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x55bd62a8d290 0x55bd62a8f6b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd62a12ad0 0x55bd62a16570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd62a08210 0x55bd62a16030 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 --2- 172.21.15.99:0/2095986730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd62a078f0 0x55bd62a1ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 >> 172.21.15.99:0/2095986730 conn(0x55bd62a03e00 msgr2=0x55bd62a0ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.124+0000 7f88a27fc640 0 client.17016 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.125+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 shutdown_connections 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.125+0000 7f88c39fd740 1 -- 172.21.15.99:0/2095986730 wait complete. 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.125+0000 7f88c39fd740 20 client.17016 trim_cache size 0 max 16384 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-05-14T17:18:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88a80b2a90 msgr2=0x7f88a80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88a80b2a90 0x7f88a80b4f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55bd62a08e90 tx=0x7f88ac00a2d0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 msgr2=0x7f88bc19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 0x7f88bc19a630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005ab0 tx=0x7f88a4003b90 comp rx=0 tx=0).stop 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 shutdown_connections 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f88a80b2a90 0x7f88a80b4f50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88bc107940 0x7f88bc1a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88bc102340 0x7f88bc19a630 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 --2- 172.21.15.99:0/2512064526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88bc101980 0x7f88bc19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 >> 172.21.15.99:0/2512064526 conn(0x7f88bc078ad0 msgr2=0x7f88bc104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 shutdown_connections 2024-05-14T17:18:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-05-14T17:18:20.126+0000 7f88c1045640 1 -- 172.21.15.99:0/2512064526 wait complete. 2024-05-14T17:18:20.375 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db / --timeout 90 --expiration 138.4 --await-for 45.0' 2024-05-14T17:18:20.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db / --timeout 90 --expiration 138.4 --await-for 45.0 2024-05-14T17:18:20.398 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:20 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3520979570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:20.400 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-14T17:18:20.671 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9e0ae7c57d244ea58049d58556d61f3b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-14T17:18:20.674 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-14T17:18:20.684 INFO:tasks.quiescer:joining Quiescers 2024-05-14T17:18:20.684 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-14T17:18:20.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:18:20.690 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:20.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:20 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3520979570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:20.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:20 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3520979570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:21.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:21 smithi099 ceph-mon[31851]: pgmap v464: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-05-14T17:18:21.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:21 smithi150 ceph-mon[36204]: pgmap v464: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-05-14T17:18:21.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:21 smithi193 ceph-mon[35962]: pgmap v464: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-05-14T17:18:21.916 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:22.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 -- 172.21.15.99:0/2263699654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b406bf10 msgr2=0x7f57b406c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:22.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 --2- 172.21.15.99:0/2263699654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b406bf10 0x7f57b406c2f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f57a40099c0 tx=0x7f57a4009cd0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 -- 172.21.15.99:0/2263699654 shutdown_connections 2024-05-14T17:18:22.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 --2- 172.21.15.99:0/2263699654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57b4108860 0x7f57b4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 --2- 172.21.15.99:0/2263699654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b406bf10 0x7f57b406c2f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 --2- 172.21.15.99:0/2263699654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57b406c8c0 0x7f57b4108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 -- 172.21.15.99:0/2263699654 >> 172.21.15.99:0/2263699654 conn(0x7f57b4078490 msgr2=0x7f57b40788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:22.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 -- 172.21.15.99:0/2263699654 shutdown_connections 2024-05-14T17:18:22.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.117+0000 7f57b99e3700 1 -- 172.21.15.99:0/2263699654 wait complete. 2024-05-14T17:18:22.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 Processor -- start 2024-05-14T17:18:22.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 -- start start 2024-05-14T17:18:22.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57b406bf10 0x7f57b41025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57b406c8c0 0x7f57b4102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 0x7f57b4103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4113750 con 0x7f57b4108860 2024-05-14T17:18:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41135d0 con 0x7f57b406c8c0 2024-05-14T17:18:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.118+0000 7f57b99e3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41138d0 con 0x7f57b406bf10 2024-05-14T17:18:22.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 0x7f57b4103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 0x7f57b4103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43976/0 (socket says 172.21.15.99:43976) 2024-05-14T17:18:22.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 -- 172.21.15.99:0/1793107461 learned_addr learned my addr 172.21.15.99:0/1793107461 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:22.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57b406bf10 0x7f57b41025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57b406c8c0 0x7f57b4102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 -- 172.21.15.99:0/1793107461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57b406bf10 msgr2=0x7f57b41025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:22.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57b406bf10 0x7f57b41025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 -- 172.21.15.99:0/1793107461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57b406c8c0 msgr2=0x7f57b4102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:22.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57b406c8c0 0x7f57b4102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 -- 172.21.15.99:0/1793107461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b41a7fe0 con 0x7f57b4108860 2024-05-14T17:18:22.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b37fe700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 0x7f57b4103020 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f57ac007ee0 tx=0x7f57ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:22.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b89e1700 1 -- 172.21.15.99:0/1793107461 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac004bb0 con 0x7f57b4108860 2024-05-14T17:18:22.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b41a82d0 con 0x7f57b4108860 2024-05-14T17:18:22.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b89e1700 1 -- 172.21.15.99:0/1793107461 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ac004d50 con 0x7f57b4108860 2024-05-14T17:18:22.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b89e1700 1 -- 172.21.15.99:0/1793107461 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac005920 con 0x7f57b4108860 2024-05-14T17:18:22.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.119+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41a87c0 con 0x7f57b4108860 2024-05-14T17:18:22.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.122+0000 7f57b89e1700 1 -- 172.21.15.99:0/1793107461 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f57ac01a020 con 0x7f57b4108860 2024-05-14T17:18:22.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57b89e1700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f57a00b2aa0 0x7f57a00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f57a00b2aa0 0x7f57a00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57b89e1700 1 -- 172.21.15.99:0/1793107461 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f57ac0e1c40 con 0x7f57b4108860 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 Processor -- start 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 -- start start 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f57a00b2aa0 0x7f57a00b4f60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009990 tx=0x7f57a400b790 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdcdb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdcd5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5634bdc33e20 con 0x5634bdcdb750 2024-05-14T17:18:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5634bdc239c0 con 0x5634bdcd5b60 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57c4279b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634bdc27a00 con 0x5634bdcdae30 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57aa7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdcd5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57aa7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdcd5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43988/0 (socket says 172.21.15.99:43988) 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.123+0000 7f57aa7fc700 1 -- 172.21.15.99:0/1242232232 learned_addr learned my addr 172.21.15.99:0/1242232232 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57b37fe700 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdcdb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57aa7fc700 1 -- 172.21.15.99:0/1242232232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 msgr2=0x5634bdcdb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdcdb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57aa7fc700 1 -- 172.21.15.99:0/1242232232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcd5b60 msgr2=0x5634bdcd5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57aa7fc700 1 -- 172.21.15.99:0/1242232232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634bdb73a60 con 0x5634bdcdb750 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdcdb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.124+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdcd5540 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f579c00aca0 tx=0x7f579c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a21ecffea15de75 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57a9ffb700 1 -- 172.21.15.99:0/1242232232 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c00fe50 con 0x5634bdcdb750 2024-05-14T17:18:22.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57a9ffb700 1 -- 172.21.15.99:0/1242232232 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579c0044e0 con 0x5634bdcdb750 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57a9ffb700 1 -- 172.21.15.99:0/1242232232 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c018690 con 0x5634bdcdb750 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 -- 172.21.15.99:0/1242232232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 msgr2=0x5634bdcd5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdcd5540 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f579c00aca0 tx=0x7f579c00f3c0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 -- 172.21.15.99:0/1242232232 shutdown_connections 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdcdb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1242232232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdcd5540 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 -- 172.21.15.99:0/1242232232 >> 172.21.15.99:0/1242232232 conn(0x5634bdc003e0 msgr2=0x5634bdccfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:22.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 -- 172.21.15.99:0/1242232232 shutdown_connections 2024-05-14T17:18:22.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.125+0000 7f57c4279b80 1 -- 172.21.15.99:0/1242232232 wait complete. 2024-05-14T17:18:22.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 -- start start 2024-05-14T17:18:22.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 Processor -- start 2024-05-14T17:18:22.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 0x5634bdcde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5634bdb2a130 con 0x5634bdcdae30 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5634bdbc5950 con 0x5634bdcdb750 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634bdce2190 con 0x5634bdcd5b60 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57c4279b80 10 client.0 _ll_register_callbacks cb 0x5634bdbba710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.126+0000 7f57aa7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 0x5634bdcde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 0x5634bdcde800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43996/0 (socket says 172.21.15.99:43996) 2024-05-14T17:18:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 -- 172.21.15.99:0/2316160072 learned_addr learned my addr 172.21.15.99:0/2316160072 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57b37fe700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 -- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 msgr2=0x5634bdce1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 -- 172.21.15.99:0/2316160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 msgr2=0x5634bdcded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcded40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 -- 172.21.15.99:0/2316160072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634bdbc2820 con 0x5634bdcdae30 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57b37fe700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 0x5634bdcde800 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f579c00ac70 tx=0x7f579c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2e38f696833bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f5792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f5792ffd700 1 -- 172.21.15.99:0/2316160072 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c026020 con 0x5634bdcdae30 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f5792ffd700 1 -- 172.21.15.99:0/2316160072 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f579c004200 con 0x5634bdcdae30 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579c0044e0 con 0x5634bdcdae30 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.127+0000 7f5792ffd700 1 -- 172.21.15.99:0/2316160072 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f579c031050 con 0x5634bdcdae30 2024-05-14T17:18:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 10 client.17034 fetch_fsmap learned FSMap version 45 2024-05-14T17:18:22.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5634bdcec8a0 con 0x5634bdcdae30 2024-05-14T17:18:22.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f5792ffd700 1 -- 172.21.15.99:0/2316160072 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579c0044e0 con 0x5634bdcdae30 2024-05-14T17:18:22.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f5792ffd700 1 -- 172.21.15.99:0/2316160072 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f579c0052b0 con 0x5634bdcdae30 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 10 client.17034 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 10 client.17034 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 20 client.17034 populate_metadata read hostname 'smithi099' 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdceee80 0x5634bdcf1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 4 client.17034 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634bdca14f0 con 0x5634bdceee80 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.128+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdceee80 0x5634bdcf1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.129+0000 7f5792ffd700 10 client.17034 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.129+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdceee80 0x5634bdcf1270 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca639a7c78dbbd94 server_cookie=cdf6ac6e438d858d in_seq=0 out_seq=0 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.129+0000 7f5792ffd700 1 -- 172.21.15.99:0/2316160072 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634bdca14f0 con 0x5634bdceee80 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.129+0000 7f5792ffd700 10 client.17034 handle_command_reply: tid=0 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.129+0000 7f57c4279b80 1 client.17034 shutdown 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 msgr2=0x5634bdcde800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 0x5634bdcde800 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f579c00ac70 tx=0x7f579c0039d0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 shutdown_connections 2024-05-14T17:18:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdceee80 0x5634bdcf1270 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 --2- 172.21.15.99:0/2316160072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdae30 0x5634bdcde800 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 >> 172.21.15.99:0/2316160072 conn(0x5634bdc003e0 msgr2=0x5634bdcca860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f5792ffd700 0 client.17034 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 shutdown_connections 2024-05-14T17:18:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 1 -- 172.21.15.99:0/2316160072 wait complete. 2024-05-14T17:18:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.130+0000 7f57c4279b80 20 client.17034 trim_cache size 0 max 16384 2024-05-14T17:18:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 Processor -- start 2024-05-14T17:18:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 -- start start 2024-05-14T17:18:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdceb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41b2760 con 0x5634bdcd5b60 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41aff90 con 0x5634bdcdb750 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57c4279b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ade00 con 0x5634bdcdae30 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44000/0 (socket says 172.21.15.99:44000) 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 -- 172.21.15.99:0/1938701289 learned_addr learned my addr 172.21.15.99:0/1938701289 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdceb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 -- 172.21.15.99:0/1938701289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 msgr2=0x5634bdceb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdceb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b37fe700 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 -- 172.21.15.99:0/1938701289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 msgr2=0x5634bdcca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57b2ffd700 1 -- 172.21.15.99:0/1938701289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634bdcdb460 con 0x5634bdcd5b60 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.150+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdceb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5540 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f57a40098e0 tx=0x7f57a4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7e6bc7abb49ce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57a97fa700 1 -- 172.21.15.99:0/1938701289 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4024ae0 con 0x5634bdcd5b60 2024-05-14T17:18:22.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57a97fa700 1 -- 172.21.15.99:0/1938701289 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a4024c80 con 0x5634bdcd5b60 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57a97fa700 1 -- 172.21.15.99:0/1938701289 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a400f6a0 con 0x5634bdcd5b60 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 -- 172.21.15.99:0/1938701289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 msgr2=0x5634bdcd5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5540 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f57a40098e0 tx=0x7f57a4023000 comp rx=0 tx=0).stop 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 -- 172.21.15.99:0/1938701289 shutdown_connections 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdb750 0x5634bdcca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcd5b60 0x5634bdcd5540 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 --2- 172.21.15.99:0/1938701289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcdae30 0x5634bdceb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 -- 172.21.15.99:0/1938701289 >> 172.21.15.99:0/1938701289 conn(0x5634bdc003e0 msgr2=0x5634bdcdbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:22.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.151+0000 7f57c4279b80 1 -- 172.21.15.99:0/1938701289 shutdown_connections 2024-05-14T17:18:22.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 -- 172.21.15.99:0/1938701289 wait complete. 2024-05-14T17:18:22.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 -- start start 2024-05-14T17:18:22.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 Processor -- start 2024-05-14T17:18:22.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdae30 0x5634bdce43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdce4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ae450 con 0x5634bdcdb750 2024-05-14T17:18:22.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41af8c0 con 0x5634bdcdae30 2024-05-14T17:18:22.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41aeb20 con 0x5634bdcd5b60 2024-05-14T17:18:22.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57c4279b80 10 client.0 _ll_register_callbacks cb 0x5634bdbba710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:18:22.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdce4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57b2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57b37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdce4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44004/0 (socket says 172.21.15.99:44004) 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.152+0000 7f57b37fe700 1 -- 172.21.15.99:0/3395470793 learned_addr learned my addr 172.21.15.99:0/3395470793 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57aa7fc700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdae30 0x5634bdce43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b37fe700 1 -- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 msgr2=0x5634bdce7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b37fe700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b37fe700 1 -- 172.21.15.99:0/3395470793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdae30 msgr2=0x5634bdce43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b37fe700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdae30 0x5634bdce43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b37fe700 1 -- 172.21.15.99:0/3395470793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634bdce4e70 con 0x5634bdcdb750 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57b37fe700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdce4930 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0a4fa0 tx=0x7f57ac0a93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d2822771768e3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f5792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f5792ffd700 1 -- 172.21.15.99:0/3395470793 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0a8870 con 0x5634bdcdb750 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f5792ffd700 1 -- 172.21.15.99:0/3395470793 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f57ac0a5510 con 0x5634bdcdb750 2024-05-14T17:18:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a4024c80 con 0x5634bdcdb750 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f5792ffd700 1 -- 172.21.15.99:0/3395470793 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f57ac0a8a30 con 0x5634bdcdb750 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57c4279b80 10 client.17040 fetch_fsmap learned FSMap version 45 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5634bdd5a0d0 con 0x5634bdcdb750 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.153+0000 7f5792ffd700 1 -- 172.21.15.99:0/3395470793 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a4024c80 con 0x5634bdcdb750 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f5792ffd700 1 -- 172.21.15.99:0/3395470793 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f57ac0dfae0 con 0x5634bdcdb750 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57c4279b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57c4279b80 10 client.17040 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:18:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57c4279b80 20 client.17040 populate_metadata read hostname 'smithi099' 2024-05-14T17:18:22.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57c4279b80 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdd5a8d0 0x5634bdd5ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:22.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57c4279b80 4 client.17040 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.4, "await_for": 45.0}] 2024-05-14T17:18:22.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.4, "await_for": 45.0}) -- 0x5634bdc35d70 con 0x5634bdd5a8d0 2024-05-14T17:18:22.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdd5a8d0 0x5634bdd5ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:22.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f5792ffd700 10 client.17040 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:22.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:22.154+0000 7f57b2ffd700 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdd5a8d0 0x5634bdd5ccf0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbc6b5a3967bd62 server_cookie=35b8cc98181da657 in_seq=0 out_seq=0 2024-05-14T17:18:23.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.181+0000 7f5792ffd700 1 -- 172.21.15.99:0/3395470793 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5634bdc35d70 con 0x5634bdd5a8d0 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.181+0000 7f5792ffd700 10 client.17040 handle_command_reply: tid=0 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.181+0000 7f57c4279b80 1 client.17040 shutdown 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 msgr2=0x5634bdce4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdce4930 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0a4fa0 tx=0x7f57ac0a93b0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 shutdown_connections 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x5634bdd5a8d0 0x5634bdd5ccf0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634bdcdb750 0x5634bdce4930 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634bdcd5b60 0x5634bdce7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f5792ffd700 0 client.17040 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 --2- 172.21.15.99:0/3395470793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634bdcdae30 0x5634bdce43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 >> 172.21.15.99:0/3395470793 conn(0x5634bdc003e0 msgr2=0x5634bdcd9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 shutdown_connections 2024-05-14T17:18:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 1 -- 172.21.15.99:0/3395470793 wait complete. 2024-05-14T17:18:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.182+0000 7f57c4279b80 20 client.17040 trim_cache size 0 max 16384 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 45, 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 46, 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": { 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "af1bf168": { 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "version": 46, 2024-05-14T17:18:23.185 INFO:teuthology.orchestra.run.smithi099.stdout: "age_ref": 0.0, 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "timeout": 90.0, 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "expiration": 138.4, 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "members": { 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "file:/": { 2024-05-14T17:18:23.186 INFO:teuthology.orchestra.run.smithi099.stdout: "excluded": false, 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "state": { 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "QUIESCED", 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:23.188 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:23.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f57a00b2aa0 msgr2=0x7f57a00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f57a00b2aa0 0x7f57a00b4f60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009990 tx=0x7f57a400b790 comp rx=0 tx=0).stop 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 msgr2=0x7f57b4103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 0x7f57b4103020 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f57ac007ee0 tx=0x7f57ac00d3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 shutdown_connections 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f57a00b2aa0 0x7f57a00b4f60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57b4108860 0x7f57b4103020 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57b406bf10 0x7f57b41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 --2- 172.21.15.99:0/1793107461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57b406c8c0 0x7f57b4102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 >> 172.21.15.99:0/1793107461 conn(0x7f57b4078490 msgr2=0x7f57b41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:23.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 shutdown_connections 2024-05-14T17:18:23.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:23.183+0000 7f57b99e3700 1 -- 172.21.15.99:0/1793107461 wait complete. 2024-05-14T17:18:23.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:23 smithi150 ceph-mon[36204]: pgmap v465: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-05-14T17:18:23.748 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:23 smithi099 ceph-mon[31851]: pgmap v465: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-05-14T17:18:23.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:23 smithi193 ceph-mon[35962]: pgmap v465: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-05-14T17:18:24.073 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'af1bf168', quiesce took 6.7 seconds. Will release after: 11.7 2024-05-14T17:18:24.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 -- 172.21.15.99:0/3493141000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1088f0 msgr2=0x7ff33c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 --2- 172.21.15.99:0/3493141000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1088f0 0x7ff33c110470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff33400b0d0 tx=0x7ff33400b3e0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 -- 172.21.15.99:0/3493141000 shutdown_connections 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 --2- 172.21.15.99:0/3493141000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1088f0 0x7ff33c110470 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 --2- 172.21.15.99:0/3493141000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1037f0 0x7ff33c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 --2- 172.21.15.99:0/3493141000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1041a0 0x7ff33c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 -- 172.21.15.99:0/3493141000 >> 172.21.15.99:0/3493141000 conn(0x7ff33c07a6a0 msgr2=0x7ff33c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:24.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 -- 172.21.15.99:0/3493141000 shutdown_connections 2024-05-14T17:18:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 -- 172.21.15.99:0/3493141000 wait complete. 2024-05-14T17:18:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 Processor -- start 2024-05-14T17:18:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 -- start start 2024-05-14T17:18:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.515+0000 7ff342d56700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 0x7ff33c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff342d56700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1041a0 0x7ff33c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff342d56700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1088f0 0x7ff33c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff342d56700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1136e0 con 0x7ff33c1037f0 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff342d56700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c113560 con 0x7ff33c1041a0 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff342d56700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c113860 con 0x7ff33c1088f0 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 0x7ff33c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 0x7ff33c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33846/0 (socket says 172.21.15.99:33846) 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 -- 172.21.15.99:0/3148258318 learned_addr learned my addr 172.21.15.99:0/3148258318 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff3412f3700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1088f0 0x7ff33c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff33bfff700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1041a0 0x7ff33c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 -- 172.21.15.99:0/3148258318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1088f0 msgr2=0x7ff33c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1088f0 0x7ff33c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 -- 172.21.15.99:0/3148258318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1041a0 msgr2=0x7ff33c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1041a0 0x7ff33c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 -- 172.21.15.99:0/3148258318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c19a040 con 0x7ff33c1037f0 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff3412f3700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1088f0 0x7ff33c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff340af2700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 0x7ff33c19f720 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff33000d980 tx=0x7ff33000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330017070 con 0x7ff33c1037f0 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff330004830 con 0x7ff33c1037f0 2024-05-14T17:18:24.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c19a330 con 0x7ff33c1037f0 2024-05-14T17:18:24.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.516+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330005440 con 0x7ff33c1037f0 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.517+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a65f0 con 0x7ff33c1037f0 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.517+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c06a230 con 0x7ff33c1037f0 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.520+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff3300055e0 con 0x7ff33c1037f0 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.520+0000 7ff339ffb700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff32c0b2a70 0x7ff32c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.520+0000 7ff33bfff700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff32c0b2a70 0x7ff32c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.520+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7ff3300e29a0 con 0x7ff33c1037f0 2024-05-14T17:18:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.521+0000 7ff33bfff700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff32c0b2a70 0x7ff32c0b4f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff328005ea0 tx=0x7ff328005e30 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:24.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.524+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff330020020 con 0x7ff33c1037f0 2024-05-14T17:18:24.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.838+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff33c02e560 con 0x7ff33c1037f0 2024-05-14T17:18:24.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.839+0000 7ff339ffb700 1 -- 172.21.15.99:0/3148258318 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11876 (secure 0 0 0) 0x7ff3300aa400 con 0x7ff33c1037f0 2024-05-14T17:18:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:18:24.841 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:18:05.015993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24463,24443,24448]},"id":1}]} 2024-05-14T17:18:24.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff32c0b2a70 msgr2=0x7ff32c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:24.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff32c0b2a70 0x7ff32c0b4f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff328005ea0 tx=0x7ff328005e30 comp rx=0 tx=0).stop 2024-05-14T17:18:24.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 msgr2=0x7ff33c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:24.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 0x7ff33c19f720 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff33000d980 tx=0x7ff33000dd40 comp rx=0 tx=0).stop 2024-05-14T17:18:24.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 shutdown_connections 2024-05-14T17:18:24.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7ff32c0b2a70 0x7ff32c0b4f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c1088f0 0x7ff33c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff33c1037f0 0x7ff33c19f720 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 --2- 172.21.15.99:0/3148258318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff33c1041a0 0x7ff33c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 >> 172.21.15.99:0/3148258318 conn(0x7ff33c07a6a0 msgr2=0x7ff33c109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 shutdown_connections 2024-05-14T17:18:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:24.842+0000 7ff342d56700 1 -- 172.21.15.99:0/3148258318 wait complete. 2024-05-14T17:18:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 45 2024-05-14T17:18:25.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:25 smithi099 ceph-mon[31851]: pgmap v466: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 22 MiB/s wr, 257 op/s 2024-05-14T17:18:25.514 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:25 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3148258318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:25.664 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24463 quiesce db --cancel --all' 2024-05-14T17:18:25.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24463 quiesce db --cancel --all 2024-05-14T17:18:25.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:25 smithi150 ceph-mon[36204]: pgmap v466: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 22 MiB/s wr, 257 op/s 2024-05-14T17:18:25.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:25 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3148258318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:25.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:25 smithi193 ceph-mon[35962]: pgmap v466: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 22 MiB/s wr, 257 op/s 2024-05-14T17:18:25.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:25 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3148258318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:25.905 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:26.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:26 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:26.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:26 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:26.794 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:26 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:27.512 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:27 smithi099 ceph-mon[31851]: pgmap v467: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 20 MiB/s wr, 237 op/s 2024-05-14T17:18:27.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:27 smithi150 ceph-mon[36204]: pgmap v467: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 20 MiB/s wr, 237 op/s 2024-05-14T17:18:27.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/2491358916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac1095b0 msgr2=0x7fd5ac109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/2491358916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac109990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b0a0 tx=0x7fd59c00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/2491358916 shutdown_connections 2024-05-14T17:18:27.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/2491358916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac06c650 0x7fd5ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/2491358916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac109990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/2491358916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/2491358916 >> 172.21.15.99:0/2491358916 conn(0x7fd5ac079410 msgr2=0x7fd5ac079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/2491358916 shutdown_connections 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/2491358916 wait complete. 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 Processor -- start 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.747+0000 7fd5b1ef9700 1 -- start start 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5b1ef9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5b1ef9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac06c650 0x7fd5ac19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5b1ef9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5b1ef9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac113740 con 0x7fd5ac06bc90 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5b1ef9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1135c0 con 0x7fd5ac1095b0 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5b1ef9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1138c0 con 0x7fd5ac06c650 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33852/0 (socket says 172.21.15.99:33852) 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/470421208 learned_addr learned my addr 172.21.15.99:0/470421208 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac06c650 0x7fd5ac19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5abfff700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/470421208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac06c650 msgr2=0x7fd5ac19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac06c650 0x7fd5ac19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/470421208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac1095b0 msgr2=0x7fd5ac19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/470421208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac19ed20 con 0x7fd5ac06bc90 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5abfff700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac199d10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b070 tx=0x7fd59c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.748+0000 7fd5a8ff9700 1 -- 172.21.15.99:0/470421208 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c021070 con 0x7fd5ac06bc90 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.749+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac19efb0 con 0x7fd5ac06bc90 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.749+0000 7fd5a8ff9700 1 -- 172.21.15.99:0/470421208 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59c007e30 con 0x7fd5ac06bc90 2024-05-14T17:18:27.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.749+0000 7fd5a8ff9700 1 -- 172.21.15.99:0/470421208 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c004840 con 0x7fd5ac06bc90 2024-05-14T17:18:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.749+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac1a6890 con 0x7fd5ac06bc90 2024-05-14T17:18:27.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.751+0000 7fd5a8ff9700 1 -- 172.21.15.99:0/470421208 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd59c007980 con 0x7fd5ac06bc90 2024-05-14T17:18:27.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5a8ff9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd5940b29f0 0x7fd5940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd5940b29f0 0x7fd5940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5a8ff9700 1 -- 172.21.15.99:0/470421208 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fd59c00f020 con 0x7fd5ac06bc90 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 Processor -- start 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 -- start start 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd5940b29f0 0x7fd5940b4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac079ad0 tx=0x7fd5a000b410 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fb4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb5270 0x563dd3faf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563dd3f08de0 con 0x563dd3fb4950 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563dd3ef8980 con 0x563dd3fb5270 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563dd3efc9c0 con 0x563dd3faf5f0 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fb4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fb4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33866/0 (socket says 172.21.15.99:33866) 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.752+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/3692712563 learned_addr learned my addr 172.21.15.99:0/3692712563 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb5270 0x563dd3faf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/3692712563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 msgr2=0x563dd3fafa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5abfff700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fafa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/3692712563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb5270 msgr2=0x563dd3faf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb5270 0x563dd3faf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/3692712563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd3ea2020 con 0x563dd3fb4950 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb5270 0x563dd3faf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5abfff700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fafa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fb4d30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0e12d0 tx=0x7fd59c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e8fb7b3a099cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:27.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd591ffb700 1 -- 172.21.15.99:0/3692712563 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0af240 con 0x563dd3fb4950 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd591ffb700 1 -- 172.21.15.99:0/3692712563 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59c0b3c60 con 0x563dd3fb4950 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.753+0000 7fd591ffb700 1 -- 172.21.15.99:0/3692712563 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0ab0f0 con 0x563dd3fb4950 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3692712563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 msgr2=0x563dd3fb4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fb4d30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0e12d0 tx=0x7fd59c0042e0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3692712563 shutdown_connections 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fafa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fb4d30 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3692712563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb5270 0x563dd3faf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3692712563 >> 172.21.15.99:0/3692712563 conn(0x563dd3ed7e20 msgr2=0x563dd3fa5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3692712563 shutdown_connections 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3692712563 wait complete. 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 -- start start 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.754+0000 7fd5bc78fb80 1 Processor -- start 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 0x563dd3fbae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb4950 0x563dd3fb8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb8770 0x563dd3fb8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563dd3dff010 con 0x563dd3faf5f0 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563dd3e60e10 con 0x563dd3fb8770 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563dd3fbb9c0 con 0x563dd3fb4950 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 10 client.0 _ll_register_callbacks cb 0x563dd3e939c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 0x563dd3fbae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 0x563dd3fbae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33874/0 (socket says 172.21.15.99:33874) 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/1299618677 learned_addr learned my addr 172.21.15.99:0/1299618677 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb4950 0x563dd3fb8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb4950 msgr2=0x563dd3fb8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb4950 0x563dd3fb8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/1299618677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb8770 msgr2=0x563dd3fb8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb8770 0x563dd3fb8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 -- 172.21.15.99:0/1299618677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd3e48940 con 0x563dd3faf5f0 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 0x563dd3fbae90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0e3c40 tx=0x7fd59c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9cd1d918b9639d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd58affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd58affd700 1 -- 172.21.15.99:0/1299618677 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c06e070 con 0x563dd3faf5f0 2024-05-14T17:18:27.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd58affd700 1 -- 172.21.15.99:0/1299618677 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fd59c0ab0f0 con 0x563dd3faf5f0 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.755+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd59c0b3c60 con 0x563dd3faf5f0 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd58affd700 1 -- 172.21.15.99:0/1299618677 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd59c0ab2f0 con 0x563dd3faf5f0 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 10 client.17058 fetch_fsmap learned FSMap version 46 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563dd3eb27d0 con 0x563dd3faf5f0 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd58affd700 1 -- 172.21.15.99:0/1299618677 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59c0b3c60 con 0x563dd3faf5f0 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd58affd700 1 -- 172.21.15.99:0/1299618677 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) ==== 8354+0+0 (secure 0 0 0) 0x7fd59c0c2040 con 0x563dd3faf5f0 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 10 client.17058 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 10 client.17058 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:18:27.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 20 client.17058 populate_metadata read hostname 'smithi099' 2024-05-14T17:18:27.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd3fc8690 0x563dd3fcaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 4 client.17058 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:18:27.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dd3f0ad30 con 0x563dd3fc8690 2024-05-14T17:18:27.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.756+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd3fc8690 0x563dd3fcaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.757+0000 7fd58affd700 10 client.17058 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:27.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.757+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd3fc8690 0x563dd3fcaa80 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c028a34c5a9f3 server_cookie=7d12e9e0552744ef in_seq=0 out_seq=0 2024-05-14T17:18:27.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd58affd700 1 -- 172.21.15.99:0/1299618677 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563dd3f0ad30 con 0x563dd3fc8690 2024-05-14T17:18:27.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd58affd700 10 client.17058 handle_command_reply: tid=0 2024-05-14T17:18:27.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 client.17058 shutdown 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 msgr2=0x563dd3fbae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 0x563dd3fbae90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0e3c40 tx=0x7fd59c0049e0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 shutdown_connections 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb8770 0x563dd3fb8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd3fc8690 0x563dd3fcaa80 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3faf5f0 0x563dd3fbae90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1299618677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb4950 0x563dd3fb8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 >> 172.21.15.99:0/1299618677 conn(0x563dd3ed7e20 msgr2=0x563dd3f9ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:27.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd58affd700 0 client.17058 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:27.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 shutdown_connections 2024-05-14T17:18:27.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1299618677 wait complete. 2024-05-14T17:18:27.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.758+0000 7fd5bc78fb80 20 client.17058 trim_cache size 0 max 16384 2024-05-14T17:18:27.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 Processor -- start 2024-05-14T17:18:27.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 -- start start 2024-05-14T17:18:27.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fa0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb4950 0x563dd3faa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 0x563dd3fd0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ae580 con 0x563dd3fb8770 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1abd80 con 0x563dd3fb4950 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1a9bf0 con 0x563dd3faf5f0 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 0x563dd3fd0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5abfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 0x563dd3fd0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33878/0 (socket says 172.21.15.99:33878) 2024-05-14T17:18:27.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5abfff700 1 -- 172.21.15.99:0/1833111972 learned_addr learned my addr 172.21.15.99:0/1833111972 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fa0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.778+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb4950 0x563dd3faa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5abfff700 1 -- 172.21.15.99:0/1833111972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 msgr2=0x563dd3fa0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5abfff700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fa0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5abfff700 1 -- 172.21.15.99:0/1833111972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb4950 msgr2=0x563dd3faa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5abfff700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb4950 0x563dd3faa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5abfff700 1 -- 172.21.15.99:0/1833111972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd3fa70e0 con 0x563dd3fb8770 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fa0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:27.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb4950 0x563dd3faa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:18:27.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5abfff700 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 0x563dd3fd0330 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097b0 tx=0x7fd5a4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5929c9b22985a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:27.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5917fa700 1 -- 172.21.15.99:0/1833111972 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4017020 con 0x563dd3fb8770 2024-05-14T17:18:27.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5917fa700 1 -- 172.21.15.99:0/1833111972 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a4013ac0 con 0x563dd3fb8770 2024-05-14T17:18:27.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5917fa700 1 -- 172.21.15.99:0/1833111972 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400f040 con 0x563dd3fb8770 2024-05-14T17:18:27.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1833111972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 msgr2=0x563dd3fd0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.779+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 0x563dd3fd0330 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097b0 tx=0x7fd5a4009ac0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1833111972 shutdown_connections 2024-05-14T17:18:27.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb8770 0x563dd3fd0330 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3faf5f0 0x563dd3fa0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/1833111972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3fb4950 0x563dd3faa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1833111972 >> 172.21.15.99:0/1833111972 conn(0x563dd3ed7e20 msgr2=0x563dd3fa9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1833111972 shutdown_connections 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/1833111972 wait complete. 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 -- start start 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 Processor -- start 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3faf5f0 0x563dd3fbdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fbe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.780+0000 7fd5bc78fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb8770 0x563dd4032ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1aa240 con 0x563dd3fb4950 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ab6b0 con 0x563dd3faf5f0 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1aa910 con 0x563dd3fb8770 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 10 client.0 _ll_register_callbacks cb 0x563dd3e939c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fbe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fbe4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33894/0 (socket says 172.21.15.99:33894) 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 -- 172.21.15.99:0/3530802892 learned_addr learned my addr 172.21.15.99:0/3530802892 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5abfff700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb8770 0x563dd4032ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3faf5f0 0x563dd3fbdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 -- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb8770 msgr2=0x563dd4032ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb8770 0x563dd4032ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 -- 172.21.15.99:0/3530802892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3faf5f0 msgr2=0x563dd3fbdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3faf5f0 0x563dd3fbdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 -- 172.21.15.99:0/3530802892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd40334e0 con 0x563dd3fb4950 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5aaffd700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fbe4c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0005d90 tx=0x7fd5a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726f1831ed805214 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd58affd700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd58affd700 1 -- 172.21.15.99:0/3530802892 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a002c060 con 0x563dd3fb4950 2024-05-14T17:18:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd58affd700 1 -- 172.21.15.99:0/3530802892 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fd5a0020b60 con 0x563dd3fb4950 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a4013ac0 con 0x563dd3fb4950 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd58affd700 1 -- 172.21.15.99:0/3530802892 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd5a0020d60 con 0x563dd3fb4950 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 10 client.17064 fetch_fsmap learned FSMap version 46 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.781+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563dd40347e0 con 0x563dd3fb4950 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd58affd700 1 -- 172.21.15.99:0/3530802892 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a4013ac0 con 0x563dd3fb4950 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd58affd700 1 -- 172.21.15.99:0/3530802892 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 46) ==== 8354+0+0 (secure 0 0 0) 0x7fd5a0027ce0 con 0x563dd3fb4950 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5bc78fb80 10 client.17064 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:18:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5bc78fb80 10 client.17064 resolve_mds: validated gid 24463 aka daemon mds.i 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5bc78fb80 20 client.17064 populate_metadata read hostname 'smithi099' 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd4034f30 0x563dd4037350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5bc78fb80 4 client.17064 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 --> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x563dd3f04510 con 0x563dd4034f30 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd4034f30 0x563dd4037350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd5ab7fe700 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd4034f30 0x563dd4037350 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc29818041803ee server_cookie=29f36ec4b20d40a1 in_seq=0 out_seq=0 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.782+0000 7fd58affd700 10 client.17064 ms_handle_connect on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd58affd700 1 -- 172.21.15.99:0/3530802892 <== mds.0 v2:172.21.15.193:6838/4034311425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x563dd3f04510 con 0x563dd4034f30 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd58affd700 10 client.17064 handle_command_reply: tid=0 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 client.17064 shutdown 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 msgr2=0x563dd3fbe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fbe4c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0005d90 tx=0x7fd5a00047c0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 shutdown_connections 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563dd3fb8770 0x563dd4032ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dd3faf5f0 0x563dd3fbdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.193:6838/4034311425,v1:172.21.15.193:6839/4034311425] conn(0x563dd4034f30 0x563dd4037350 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 --2- 172.21.15.99:0/3530802892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563dd3fb4950 0x563dd3fbe4c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd58affd700 0 client.17064 ms_handle_reset on v2:172.21.15.193:6838/4034311425 2024-05-14T17:18:27.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 >> 172.21.15.99:0/3530802892 conn(0x563dd3ed7e20 msgr2=0x563dd3fb5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:27.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 shutdown_connections 2024-05-14T17:18:27.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 1 -- 172.21.15.99:0/3530802892 wait complete. 2024-05-14T17:18:27.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.783+0000 7fd5bc78fb80 20 client.17064 trim_cache size 0 max 16384 2024-05-14T17:18:27.794 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 46, 2024-05-14T17:18:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "leader": 24463, 2024-05-14T17:18:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "set_version": 47, 2024-05-14T17:18:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "sets": {} 2024-05-14T17:18:27.795 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd5940b29f0 msgr2=0x7fd5940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd5940b29f0 0x7fd5940b4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac079ad0 tx=0x7fd5a000b410 comp rx=0 tx=0).stop 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 msgr2=0x7fd5ac199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac199d10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b070 tx=0x7fd59c004d00 comp rx=0 tx=0).stop 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 shutdown_connections 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fd5940b29f0 0x7fd5940b4eb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac1095b0 0x7fd5ac19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5ac06bc90 0x7fd5ac199d10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 --2- 172.21.15.99:0/470421208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac06c650 0x7fd5ac19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:27.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 >> 172.21.15.99:0/470421208 conn(0x7fd5ac079410 msgr2=0x7fd5ac1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:27.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 shutdown_connections 2024-05-14T17:18:27.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:27.784+0000 7fd5b1ef9700 1 -- 172.21.15.99:0/470421208 wait complete. 2024-05-14T17:18:27.797 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:27 smithi193 ceph-mon[35962]: pgmap v467: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 20 MiB/s wr, 237 op/s 2024-05-14T17:18:28.572 INFO:tasks.quiescer:done joining Quiescers 2024-05-14T17:18:28.573 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-14T17:18:28.583 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:18:28.592 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-14T17:18:28.603 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-14T17:18:28.603 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T17:18:28.603 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:18:28.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-14T17:18:28.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-14T17:18:28.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T17:18:28.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 -- 172.21.15.99:0/73568540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 msgr2=0x7faddc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:28.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 --2- 172.21.15.99:0/73568540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc102ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009930 tx=0x7fadcc02a420 comp rx=0 tx=0).stop 2024-05-14T17:18:28.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 -- 172.21.15.99:0/73568540 shutdown_connections 2024-05-14T17:18:28.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 --2- 172.21.15.99:0/73568540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faddc103410 0x7faddc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:28.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 --2- 172.21.15.99:0/73568540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc102ed0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:28.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 --2- 172.21.15.99:0/73568540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc108a60 0x7faddc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 -- 172.21.15.99:0/73568540 >> 172.21.15.99:0/73568540 conn(0x7faddc0fe520 msgr2=0x7faddc100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 -- 172.21.15.99:0/73568540 shutdown_connections 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 -- 172.21.15.99:0/73568540 wait complete. 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 Processor -- start 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 -- start start 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:28.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.877+0000 7fade2def640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103410 0x7faddc075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade2def640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faddc108a60 0x7faddc0757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade2def640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faddc112d20 con 0x7faddc102a50 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade2def640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faddc112ba0 con 0x7faddc108a60 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade2def640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faddc112ea0 con 0x7faddc103410 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33914/0 (socket says 172.21.15.99:33914) 2024-05-14T17:18:28.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 -- 172.21.15.99:0/3322691936 learned_addr learned my addr 172.21.15.99:0/3322691936 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade138c640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faddc108a60 0x7faddc0757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fadd3fff640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103410 0x7faddc075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 -- 172.21.15.99:0/3322691936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103410 msgr2=0x7faddc075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103410 0x7faddc075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 -- 172.21.15.99:0/3322691936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faddc108a60 msgr2=0x7faddc0757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faddc108a60 0x7faddc0757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:28.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 -- 172.21.15.99:0/3322691936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a7700 con 0x7faddc102a50 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fadd3fff640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103410 0x7faddc075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade0b8b640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc074d50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fadd800d770 tx=0x7fadd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8017070 con 0x7faddc102a50 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd80054d0 con 0x7faddc102a50 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a79f0 con 0x7faddc102a50 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8004c30 con 0x7faddc102a50 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.878+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a7f40 con 0x7faddc102a50 2024-05-14T17:18:28.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.879+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc0507e0 con 0x7faddc102a50 2024-05-14T17:18:28.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.880+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fadd8004df0 con 0x7faddc102a50 2024-05-14T17:18:28.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.881+0000 7fadd1ffb640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fadc40b2a90 0x7fadc40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:28.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.882+0000 7fadd3fff640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fadc40b2a90 0x7fadc40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:28.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.882+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fadd80e20d0 con 0x7faddc102a50 2024-05-14T17:18:28.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.882+0000 7fadd3fff640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fadc40b2a90 0x7fadc40b4f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadcc000c00 tx=0x7fadcc031400 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:28.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:28.886+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadd80a9ab0 con 0x7faddc102a50 2024-05-14T17:18:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.089+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7faddc062e90 con 0x7fadc40b2a90 2024-05-14T17:18:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.097+0000 7fadd1ffb640 1 -- 172.21.15.99:0/3322691936 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7faddc062e90 con 0x7fadc40b2a90 2024-05-14T17:18:29.097 INFO:teuthology.orchestra.run.smithi099.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T17:03:49", "first": "2024-05-14T17:04:00", "last": "2024-05-14T17:18:00", "last_pruned": "2024-05-14T17:18:00", "created_count": 15, "pruned_count": 9, "active": true} 2024-05-14T17:18:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fadc40b2a90 msgr2=0x7fadc40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fadc40b2a90 0x7fadc40b4f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadcc000c00 tx=0x7fadcc031400 comp rx=0 tx=0).stop 2024-05-14T17:18:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 msgr2=0x7faddc074d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc074d50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fadd800d770 tx=0x7fadd800dc30 comp rx=0 tx=0).stop 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 shutdown_connections 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fadc40b2a90 0x7fadc40b4f50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faddc108a60 0x7faddc0757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faddc103410 0x7faddc075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 --2- 172.21.15.99:0/3322691936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faddc102a50 0x7faddc074d50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.099+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 >> 172.21.15.99:0/3322691936 conn(0x7faddc0fe520 msgr2=0x7faddc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.100+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 shutdown_connections 2024-05-14T17:18:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.100+0000 7fade2def640 1 -- 172.21.15.99:0/3322691936 wait complete. 2024-05-14T17:18:29.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T17:18:29.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 -- 172.21.15.99:0/3160924978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0102a60 msgr2=0x7f92c0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3160924978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0102a60 0x7f92c0102ee0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009930 tx=0x7f92a802a420 comp rx=0 tx=0).stop 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 -- 172.21.15.99:0/3160924978 shutdown_connections 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3160924978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c0103420 0x7f92c010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3160924978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0102a60 0x7f92c0102ee0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3160924978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0108a70 0x7f92c0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 -- 172.21.15.99:0/3160924978 >> 172.21.15.99:0/3160924978 conn(0x7f92c0079ee0 msgr2=0x7f92c007a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:29.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 -- 172.21.15.99:0/3160924978 shutdown_connections 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.213+0000 7f92c5a83640 1 -- 172.21.15.99:0/3160924978 wait complete. 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 Processor -- start 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 -- start start 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0102a60 0x7f92c0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c0103420 0x7f92c0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 0x7f92c019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0112d30 con 0x7f92c0108a70 2024-05-14T17:18:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0112bb0 con 0x7f92c0102a60 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92c5a83640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0112eb0 con 0x7f92c0103420 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 0x7f92c019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 0x7f92c019db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33940/0 (socket says 172.21.15.99:33940) 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 -- 172.21.15.99:0/3871859167 learned_addr learned my addr 172.21.15.99:0/3871859167 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bf7fe640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0102a60 0x7f92c0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92b7fff640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c0103420 0x7f92c0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 -- 172.21.15.99:0/3871859167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c0103420 msgr2=0x7f92c0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c0103420 0x7f92c0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 -- 172.21.15.99:0/3871859167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0102a60 msgr2=0x7f92c0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0102a60 0x7f92c0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 -- 172.21.15.99:0/3871859167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c019e240 con 0x7f92c0108a70 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bffff640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 0x7f92c019db10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b9e0 tx=0x7f92b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.214+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8019070 con 0x7f92c0108a70 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.215+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b8014440 con 0x7f92c0108a70 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.215+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c019e530 con 0x7f92c0108a70 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.215+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80075d0 con 0x7f92c0108a70 2024-05-14T17:18:29.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.215+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c01a5dc0 con 0x7f92c0108a70 2024-05-14T17:18:29.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.216+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c00507e0 con 0x7f92c0108a70 2024-05-14T17:18:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.217+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f92b801c070 con 0x7f92c0108a70 2024-05-14T17:18:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.218+0000 7f92bd7fa640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92ac0b2a10 0x7f92ac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.218+0000 7f92bf7fe640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92ac0b2a10 0x7f92ac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.218+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f92b80e1ac0 con 0x7f92c0108a70 2024-05-14T17:18:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.218+0000 7f92bf7fe640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92ac0b2a10 0x7f92ac0b4ed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f92b000bbe0 tx=0x7f92b000b390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:29.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.222+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92b80a8ec0 con 0x7f92c0108a70 2024-05-14T17:18:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.429+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 --> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f92c010b060 con 0x7f92ac0b2a10 2024-05-14T17:18:29.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.438+0000 7f92bd7fa640 1 -- 172.21.15.99:0/3871859167 <== mgr.24710 v2:172.21.15.99:6800/2715605784 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f92b0010c30 con 0x7f92ac0b2a10 2024-05-14T17:18:29.437 INFO:teuthology.orchestra.run.smithi099.stdout:/ 1m 6m3h 2024-05-14T17:18:29.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92ac0b2a10 msgr2=0x7f92ac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92ac0b2a10 0x7f92ac0b4ed0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f92b000bbe0 tx=0x7f92b000b390 comp rx=0 tx=0).stop 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 msgr2=0x7f92c019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 0x7f92c019db10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b9e0 tx=0x7f92b800bea0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 shutdown_connections 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f92ac0b2a10 0x7f92ac0b4ed0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c0108a70 0x7f92c019db10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c0103420 0x7f92c0199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 --2- 172.21.15.99:0/3871859167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92c0102a60 0x7f92c0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 >> 172.21.15.99:0/3871859167 conn(0x7f92c0079ee0 msgr2=0x7f92c01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:29.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 shutdown_connections 2024-05-14T17:18:29.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.440+0000 7f92c5a83640 1 -- 172.21.15.99:0/3871859167 wait complete. 2024-05-14T17:18:29.451 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-14T17:18:29.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T17:18:29.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 -- 172.21.15.99:0/3756777411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c88101930 msgr2=0x7f7c88101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 --2- 172.21.15.99:0/3756777411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c88101930 0x7f7c88101db0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009930 tx=0x7f7c8402a420 comp rx=0 tx=0).stop 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 -- 172.21.15.99:0/3756777411 shutdown_connections 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 --2- 172.21.15.99:0/3756777411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c881022f0 0x7f7c8810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 --2- 172.21.15.99:0/3756777411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c88101930 0x7f7c88101db0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 --2- 172.21.15.99:0/3756777411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c88107940 0x7f7c88107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 -- 172.21.15.99:0/3756777411 >> 172.21.15.99:0/3756777411 conn(0x7f7c88078a50 msgr2=0x7f7c88078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 -- 172.21.15.99:0/3756777411 shutdown_connections 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 -- 172.21.15.99:0/3756777411 wait complete. 2024-05-14T17:18:29.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 Processor -- start 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.565+0000 7f7c90522640 1 -- start start 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c90522640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c88101930 0x7f7c88198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c90522640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 0x7f7c88198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c90522640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c88107940 0x7f7c8819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c90522640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111c00 con 0x7f7c881022f0 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c90522640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111a80 con 0x7f7c88101930 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c90522640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88111d80 con 0x7f7c88107940 2024-05-14T17:18:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 0x7f7c88198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 0x7f7c88198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33964/0 (socket says 172.21.15.99:33964) 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 -- 172.21.15.99:0/273565545 learned_addr learned my addr 172.21.15.99:0/273565545 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8e2be640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c88101930 0x7f7c88198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8eabf640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c88107940 0x7f7c8819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 -- 172.21.15.99:0/273565545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c88107940 msgr2=0x7f7c8819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c88107940 0x7f7c8819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 -- 172.21.15.99:0/273565545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c88101930 msgr2=0x7f7c88198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c88101930 0x7f7c88198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 -- 172.21.15.99:0/273565545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8819d1e0 con 0x7f7c881022f0 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8e2be640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c88101930 0x7f7c88198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.566+0000 7f7c8dabd640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 0x7f7c88198780 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004770 tx=0x7f7c84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.567+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84035050 con 0x7f7c881022f0 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.567+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c84003aa0 con 0x7f7c881022f0 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.567+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8819d470 con 0x7f7c881022f0 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.567+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84038ae0 con 0x7f7c881022f0 2024-05-14T17:18:29.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.567+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c88069be0 con 0x7f7c881022f0 2024-05-14T17:18:29.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.568+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c88103070 con 0x7f7c881022f0 2024-05-14T17:18:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.570+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7c84005020 con 0x7f7c881022f0 2024-05-14T17:18:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.571+0000 7f7c7b7fe640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7c740b2960 0x7f7c740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.571+0000 7f7c8e2be640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7c740b2960 0x7f7c740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.571+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f7c84104b50 con 0x7f7c881022f0 2024-05-14T17:18:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.572+0000 7f7c8e2be640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7c740b2960 0x7f7c740b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c005fd0 tx=0x7f7c7c00b390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:29.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.575+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c840cc530 con 0x7f7c881022f0 2024-05-14T17:18:29.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:29 smithi150 ceph-mon[36204]: pgmap v468: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 18 MiB/s wr, 214 op/s 2024-05-14T17:18:29.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.762+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c8806a6c0 con 0x7f7c881022f0 2024-05-14T17:18:29.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.764+0000 7f7c7b7fe640 1 -- 172.21.15.99:0/273565545 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23165 (secure 0 0 0) 0x7f7c840cbe30 con 0x7f7c881022f0 2024-05-14T17:18:29.764 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:18:29.764 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":103,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:18:14.954778+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/2498052825":"2024-05-15T17:03:37.568496+0000","172.21.15.99:6800/2498052825":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/1935223933":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/781830097":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/701736301":"2024-05-15T17:03:37.568496+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2842786850":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:18:29.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:29 smithi099 ceph-mon[31851]: pgmap v468: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 18 MiB/s wr, 214 op/s 2024-05-14T17:18:29.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7c740b2960 msgr2=0x7f7c740b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7c740b2960 0x7f7c740b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c005fd0 tx=0x7f7c7c00b390 comp rx=0 tx=0).stop 2024-05-14T17:18:29.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 msgr2=0x7f7c88198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 0x7f7c88198780 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004770 tx=0x7f7c84004270 comp rx=0 tx=0).stop 2024-05-14T17:18:29.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 shutdown_connections 2024-05-14T17:18:29.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7c740b2960 0x7f7c740b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c88107940 0x7f7c8819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c881022f0 0x7f7c88198780 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 --2- 172.21.15.99:0/273565545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c88101930 0x7f7c88198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 >> 172.21.15.99:0/273565545 conn(0x7f7c88078a50 msgr2=0x7f7c88104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 shutdown_connections 2024-05-14T17:18:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.766+0000 7f7c90522640 1 -- 172.21.15.99:0/273565545 wait complete. 2024-05-14T17:18:29.777 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T16:55:26.450956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T17:00:05.990282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T17:00:08.701546+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T17:18:29.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T17:18:29.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:29 smithi193 ceph-mon[35962]: pgmap v468: 97 pgs: 97 active+clean; 403 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 18 MiB/s wr, 214 op/s 2024-05-14T17:18:29.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 -- 172.21.15.99:0/802455910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 msgr2=0x7f7acc10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 --2- 172.21.15.99:0/802455910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc10e800 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009cd0 tx=0x7f7ac402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 -- 172.21.15.99:0/802455910 shutdown_connections 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 --2- 172.21.15.99:0/802455910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc10e800 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 --2- 172.21.15.99:0/802455910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7acc101930 0x7f7acc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 --2- 172.21.15.99:0/802455910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7acc107940 0x7f7acc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 -- 172.21.15.99:0/802455910 >> 172.21.15.99:0/802455910 conn(0x7f7acc0789b0 msgr2=0x7f7acc078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 -- 172.21.15.99:0/802455910 shutdown_connections 2024-05-14T17:18:29.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 -- 172.21.15.99:0/802455910 wait complete. 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 Processor -- start 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.913+0000 7f7ad246b640 1 -- start start 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad246b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7acc101930 0x7f7acc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad246b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad246b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7acc107940 0x7f7acc19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad246b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc111c00 con 0x7f7acc1022f0 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad246b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc111a80 con 0x7f7acc101930 2024-05-14T17:18:29.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad246b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc111d80 con 0x7f7acc107940 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33980/0 (socket says 172.21.15.99:33980) 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 -- 172.21.15.99:0/1909599696 learned_addr learned my addr 172.21.15.99:0/1909599696 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ad0a08640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7acc107940 0x7f7acc19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acbfff640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7acc101930 0x7f7acc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 -- 172.21.15.99:0/1909599696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7acc107940 msgr2=0x7f7acc19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7acc107940 0x7f7acc19db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 -- 172.21.15.99:0/1909599696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7acc101930 msgr2=0x7f7acc1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:29.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7acc101930 0x7f7acc1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:29.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 -- 172.21.15.99:0/1909599696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc19e250 con 0x7f7acc1022f0 2024-05-14T17:18:29.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7acb7fe640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc1997f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400d950 tx=0x7f7ab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:29.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.914+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4017070 con 0x7f7acc1022f0 2024-05-14T17:18:29.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.915+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc19e540 con 0x7f7acc1022f0 2024-05-14T17:18:29.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.915+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc1a5dd0 con 0x7f7acc1022f0 2024-05-14T17:18:29.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.915+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab4009c30 con 0x7f7acc1022f0 2024-05-14T17:18:29.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.915+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4020060 con 0x7f7acc1022f0 2024-05-14T17:18:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.916+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc069cc0 con 0x7f7acc1022f0 2024-05-14T17:18:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.917+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7ab4010460 con 0x7f7acc1022f0 2024-05-14T17:18:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.918+0000 7f7ac97fa640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7ab80b2960 0x7f7ab80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.918+0000 7f7acbfff640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7ab80b2960 0x7f7ab80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.918+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f7ab40e1e40 con 0x7f7acc1022f0 2024-05-14T17:18:29.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.918+0000 7f7acbfff640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7ab80b2960 0x7f7ab80b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009b50 tx=0x7f7abc005f90 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:29.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:29.922+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ab40a91d0 con 0x7f7acc1022f0 2024-05-14T17:18:30.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.137+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7acc02dba0 con 0x7f7acc1022f0 2024-05-14T17:18:30.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.138+0000 7f7ac97fa640 1 -- 172.21.15.99:0/1909599696 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f7ab40ad4d0 con 0x7f7acc1022f0 2024-05-14T17:18:30.138 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-05-14T17:18:30.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7ab80b2960 msgr2=0x7f7ab80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7ab80b2960 0x7f7ab80b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009b50 tx=0x7f7abc005f90 comp rx=0 tx=0).stop 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 msgr2=0x7f7acc1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc1997f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400d950 tx=0x7f7ab400de10 comp rx=0 tx=0).stop 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 shutdown_connections 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f7ab80b2960 0x7f7ab80b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7acc107940 0x7f7acc19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7acc1022f0 0x7f7acc1997f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 --2- 172.21.15.99:0/1909599696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7acc101930 0x7f7acc1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 >> 172.21.15.99:0/1909599696 conn(0x7f7acc0789b0 msgr2=0x7f7acc06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:30.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 shutdown_connections 2024-05-14T17:18:30.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.140+0000 7f7ad246b640 1 -- 172.21.15.99:0/1909599696 wait complete. 2024-05-14T17:18:30.151 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T17:18:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 -- 172.21.15.99:0/1783778278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c102a20 msgr2=0x7fa43c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 --2- 172.21.15.99:0/1783778278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c102a20 0x7fa43c102ea0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa430009930 tx=0x7fa43002a420 comp rx=0 tx=0).stop 2024-05-14T17:18:30.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 -- 172.21.15.99:0/1783778278 shutdown_connections 2024-05-14T17:18:30.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 --2- 172.21.15.99:0/1783778278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43c1033e0 0x7fa43c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 --2- 172.21.15.99:0/1783778278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c102a20 0x7fa43c102ea0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 --2- 172.21.15.99:0/1783778278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa43c108a30 0x7fa43c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.255+0000 7fa4443da640 1 -- 172.21.15.99:0/1783778278 >> 172.21.15.99:0/1783778278 conn(0x7fa43c0fe490 msgr2=0x7fa43c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:30.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 -- 172.21.15.99:0/1783778278 shutdown_connections 2024-05-14T17:18:30.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 -- 172.21.15.99:0/1783778278 wait complete. 2024-05-14T17:18:30.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 Processor -- start 2024-05-14T17:18:30.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 -- start start 2024-05-14T17:18:30.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43c102a20 0x7fa43c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa43c1033e0 0x7fa43c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 0x7fa43c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c112cf0 con 0x7fa43c108a30 2024-05-14T17:18:30.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c112b70 con 0x7fa43c102a20 2024-05-14T17:18:30.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa4443da640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c112e70 con 0x7fa43c1033e0 2024-05-14T17:18:30.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.256+0000 7fa442977640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 0x7fa43c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 0x7fa43c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34002/0 (socket says 172.21.15.99:34002) 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 -- 172.21.15.99:0/393965112 learned_addr learned my addr 172.21.15.99:0/393965112 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa441975640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa43c1033e0 0x7fa43c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442176640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43c102a20 0x7fa43c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 -- 172.21.15.99:0/393965112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa43c1033e0 msgr2=0x7fa43c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa43c1033e0 0x7fa43c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 -- 172.21.15.99:0/393965112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43c102a20 msgr2=0x7fa43c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43c102a20 0x7fa43c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 -- 172.21.15.99:0/393965112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c19e2a0 con 0x7fa43c108a30 2024-05-14T17:18:30.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa442977640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 0x7fa43c19db70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa43400cad0 tx=0x7fa43400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:30.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400ae20 con 0x7fa43c108a30 2024-05-14T17:18:30.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c19e590 con 0x7fa43c108a30 2024-05-14T17:18:30.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa434005020 con 0x7fa43c108a30 2024-05-14T17:18:30.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400f6b0 con 0x7fa43c108a30 2024-05-14T17:18:30.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.257+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1a5e20 con 0x7fa43c108a30 2024-05-14T17:18:30.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.258+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c069cc0 con 0x7fa43c108a30 2024-05-14T17:18:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.260+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa4340051d0 con 0x7fa43c108a30 2024-05-14T17:18:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.260+0000 7fa43b7fe640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa4240b2960 0x7fa4240b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.260+0000 7fa442176640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa4240b2960 0x7fa4240b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.260+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa4340e29a0 con 0x7fa43c108a30 2024-05-14T17:18:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.261+0000 7fa442176640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa4240b2960 0x7fa4240b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00bb60 tx=0x7fa42c00b390 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:30.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.264+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa434014030 con 0x7fa43c108a30 2024-05-14T17:18:30.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.452+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa43c02dba0 con 0x7fa43c108a30 2024-05-14T17:18:30.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.452+0000 7fa43b7fe640 1 -- 172.21.15.99:0/393965112 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fa4340a9d30 con 0x7fa43c108a30 2024-05-14T17:18:30.452 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-05-14T17:18:30.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa4240b2960 msgr2=0x7fa4240b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa4240b2960 0x7fa4240b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00bb60 tx=0x7fa42c00b390 comp rx=0 tx=0).stop 2024-05-14T17:18:30.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 msgr2=0x7fa43c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 0x7fa43c19db70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa43400cad0 tx=0x7fa43400cf90 comp rx=0 tx=0).stop 2024-05-14T17:18:30.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 shutdown_connections 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa4240b2960 0x7fa4240b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa43c108a30 0x7fa43c19db70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa43c1033e0 0x7fa43c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 --2- 172.21.15.99:0/393965112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa43c102a20 0x7fa43c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 >> 172.21.15.99:0/393965112 conn(0x7fa43c0fe490 msgr2=0x7fa43c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 shutdown_connections 2024-05-14T17:18:30.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.454+0000 7fa4443da640 1 -- 172.21.15.99:0/393965112 wait complete. 2024-05-14T17:18:30.464 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:30 smithi099 ceph-mon[31851]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:18:30.464 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/273565545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:18:30.464 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:30 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1909599696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:18:30.465 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T17:18:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 -- 172.21.15.99:0/2006284896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94106920 msgr2=0x7fbe94106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 --2- 172.21.15.99:0/2006284896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94106920 0x7fbe94106d00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009930 tx=0x7fbe8402a420 comp rx=0 tx=0).stop 2024-05-14T17:18:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 -- 172.21.15.99:0/2006284896 shutdown_connections 2024-05-14T17:18:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 --2- 172.21.15.99:0/2006284896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe94102e50 0x7fbe9410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 --2- 172.21.15.99:0/2006284896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94102490 0x7fbe94102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 --2- 172.21.15.99:0/2006284896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94106920 0x7fbe94106d00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 -- 172.21.15.99:0/2006284896 >> 172.21.15.99:0/2006284896 conn(0x7fbe94076be0 msgr2=0x7fbe94076ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 -- 172.21.15.99:0/2006284896 shutdown_connections 2024-05-14T17:18:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.570+0000 7fbe9881d640 1 -- 172.21.15.99:0/2006284896 wait complete. 2024-05-14T17:18:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 Processor -- start 2024-05-14T17:18:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 -- start start 2024-05-14T17:18:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe94102490 0x7fbe94198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 0x7fbe94198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94106920 0x7fbe9419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94111c00 con 0x7fbe94102e50 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94111a80 con 0x7fbe94106920 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9881d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94111d80 con 0x7fbe94102490 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 0x7fbe94198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 0x7fbe94198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34018/0 (socket says 172.21.15.99:34018) 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe8bfff640 1 -- 172.21.15.99:0/4177822463 learned_addr learned my addr 172.21.15.99:0/4177822463 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe92d9d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94106920 0x7fbe9419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.571+0000 7fbe9259c640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe94102490 0x7fbe94198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8bfff640 1 -- 172.21.15.99:0/4177822463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe94102490 msgr2=0x7fbe94198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8bfff640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe94102490 0x7fbe94198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8bfff640 1 -- 172.21.15.99:0/4177822463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94106920 msgr2=0x7fbe9419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8bfff640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94106920 0x7fbe9419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8bfff640 1 -- 172.21.15.99:0/4177822463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9419d130 con 0x7fbe94102e50 2024-05-14T17:18:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8bfff640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 0x7fbe94198680 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d950 tx=0x7fbe7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c017030 con 0x7fbe94102e50 2024-05-14T17:18:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9419d420 con 0x7fbe94102e50 2024-05-14T17:18:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7c004020 con 0x7fbe94102e50 2024-05-14T17:18:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0047a0 con 0x7fbe94102e50 2024-05-14T17:18:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.572+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941a4cb0 con 0x7fbe94102e50 2024-05-14T17:18:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.573+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94069cc0 con 0x7fbe94102e50 2024-05-14T17:18:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.574+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbe7c009a30 con 0x7fbe94102e50 2024-05-14T17:18:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.575+0000 7fbe8b7fe640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbe800b2960 0x7fbe800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.575+0000 7fbe9259c640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbe800b2960 0x7fbe800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.575+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fbe7c0e20f0 con 0x7fbe94102e50 2024-05-14T17:18:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.576+0000 7fbe9259c640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbe800b2960 0x7fbe800b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006010 tx=0x7fbe84009610 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.579+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe7c0a9ad0 con 0x7fbe94102e50 2024-05-14T17:18:30.702 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:30 smithi150 ceph-mon[36204]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:18:30.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:30 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/273565545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:18:30.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:30 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1909599696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:18:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.761+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbe9402dba0 con 0x7fbe94102e50 2024-05-14T17:18:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.761+0000 7fbe8b7fe640 1 -- 172.21.15.99:0/4177822463 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7fbe7c0a93d0 con 0x7fbe94102e50 2024-05-14T17:18:30.761 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-05-14T17:18:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.763+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbe800b2960 msgr2=0x7fbe800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.763+0000 7fbe9881d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbe800b2960 0x7fbe800b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006010 tx=0x7fbe84009610 comp rx=0 tx=0).stop 2024-05-14T17:18:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 msgr2=0x7fbe94198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 0x7fbe94198680 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d950 tx=0x7fbe7c00de10 comp rx=0 tx=0).stop 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 shutdown_connections 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fbe800b2960 0x7fbe800b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe94106920 0x7fbe9419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe94102e50 0x7fbe94198680 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 --2- 172.21.15.99:0/4177822463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe94102490 0x7fbe94198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 >> 172.21.15.99:0/4177822463 conn(0x7fbe94076be0 msgr2=0x7fbe940ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 shutdown_connections 2024-05-14T17:18:30.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.764+0000 7fbe9881d640 1 -- 172.21.15.99:0/4177822463 wait complete. 2024-05-14T17:18:30.775 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-14T17:18:30.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:30 smithi193 ceph-mon[35962]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:18:30.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:30 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/273565545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:18:30.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:30 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1909599696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:18:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 -- 172.21.15.99:0/1092710872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c107940 msgr2=0x7f820c107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 --2- 172.21.15.99:0/1092710872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c107940 0x7f820c107d20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8208009930 tx=0x7f820802a420 comp rx=0 tx=0).stop 2024-05-14T17:18:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 -- 172.21.15.99:0/1092710872 shutdown_connections 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 --2- 172.21.15.99:0/1092710872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c1022f0 0x7f820c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 --2- 172.21.15.99:0/1092710872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c101930 0x7f820c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 --2- 172.21.15.99:0/1092710872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c107940 0x7f820c107d20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 -- 172.21.15.99:0/1092710872 >> 172.21.15.99:0/1092710872 conn(0x7f820c078a50 msgr2=0x7f820c078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 -- 172.21.15.99:0/1092710872 shutdown_connections 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.877+0000 7f8213c50640 1 -- 172.21.15.99:0/1092710872 wait complete. 2024-05-14T17:18:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 Processor -- start 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 -- start start 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 0x7f820c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1022f0 0x7f820c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c107940 0x7f820c1a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111bd0 con 0x7f820c101930 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111a50 con 0x7f820c107940 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111d50 con 0x7f820c1022f0 2024-05-14T17:18:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 0x7f820c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 0x7f820c19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34034/0 (socket says 172.21.15.99:34034) 2024-05-14T17:18:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 -- 172.21.15.99:0/4182272714 learned_addr learned my addr 172.21.15.99:0/4182272714 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:18:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82111eb640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1022f0 0x7f820c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 -- 172.21.15.99:0/4182272714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1022f0 msgr2=0x7f820c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1022f0 0x7f820c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 -- 172.21.15.99:0/4182272714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c107940 msgr2=0x7f820c1a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c107940 0x7f820c1a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 -- 172.21.15.99:0/4182272714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c1a5610 con 0x7f820c101930 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82111eb640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1022f0 0x7f820c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f82119ec640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 0x7f820c19a050 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8208004770 tx=0x7f82080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820803f070 con 0x7f820c101930 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f820803b460 con 0x7f820c101930 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.878+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c1a57e0 con 0x7f820c101930 2024-05-14T17:18:30.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.879+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208035070 con 0x7f820c101930 2024-05-14T17:18:30.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.879+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c1a5ca0 con 0x7f820c101930 2024-05-14T17:18:30.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.880+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820c069870 con 0x7f820c101930 2024-05-14T17:18:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.881+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f820803b8d0 con 0x7f820c101930 2024-05-14T17:18:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.881+0000 7f8202ffd640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81f80b29e0 0x7f81f80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.882+0000 7f82111eb640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81f80b29e0 0x7f81f80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.882+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f8208102c00 con 0x7f820c101930 2024-05-14T17:18:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.882+0000 7f82111eb640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81f80b29e0 0x7f81f80b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f820c06a730 tx=0x7f81fc005e70 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:30.885+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82080ca5e0 con 0x7f820c101930 2024-05-14T17:18:31.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.117+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f820c02dba0 con 0x7f820c101930 2024-05-14T17:18:31.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.125+0000 7f8202ffd640 1 -- 172.21.15.99:0/4182272714 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816173 (secure 0 0 0) 0x7f82080c9ee0 con 0x7f820c101930 2024-05-14T17:18:31.136 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:18:31.136 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:31.136 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 26, 2024-05-14T17:18:31.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.136 INFO:teuthology.orchestra.run.smithi099.stdout: "active_gid": 24710, 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: "active_name": "x", 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addrs": { 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2715605784 2024-05-14T17:18:31.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2715605784 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.138 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addr": "172.21.15.99:6801/2715605784", 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "active_change": "2024-05-14T17:03:37.568710+0000", 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "active_mgr_features": 4540701547738038271, 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "available": true, 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "standbys": [ 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 24704, 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "y", 2024-05-14T17:18:31.139 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540701547738038271, 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:31.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.142 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.142 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.142 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.142 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.143 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.145 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.146 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.147 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.148 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.149 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.150 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.151 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.152 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-05-14T17:18:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-05-14T17:18:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.153 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.154 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:18:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.155 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-05-14T17:18:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-05-14T17:18:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-05-14T17:18:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.157 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-05-14T17:18:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.158 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-05-14T17:18:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-05-14T17:18:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.159 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-05-14T17:18:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-05-14T17:18:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-05-14T17:18:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.161 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.163 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.164 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.165 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-05-14T17:18:31.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.167 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.168 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.169 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.170 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.171 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.172 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.173 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-05-14T17:18:31.174 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-05-14T17:18:31.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-05-14T17:18:31.176 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:18:31.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-05-14T17:18:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-05-14T17:18:31.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:18:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.180 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.181 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:18:31.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.183 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-05-14T17:18:31.185 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.186 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.187 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.189 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.190 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.191 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.192 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.194 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.195 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-05-14T17:18:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-05-14T17:18:31.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.197 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:18:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.198 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-05-14T17:18:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-05-14T17:18:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-05-14T17:18:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.200 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "read", 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap", 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap-read" 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.201 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-05-14T17:18:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:18:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:18:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-05-14T17:18:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-05-14T17:18:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-05-14T17:18:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-05-14T17:18:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:18:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-05-14T17:18:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-05-14T17:18:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.217 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-05-14T17:18:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:18:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-05-14T17:18:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-05-14T17:18:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:18:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:18:31.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-05-14T17:18:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-05-14T17:18:31.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:31.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:18:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.225 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:18:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:18:31.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-05-14T17:18:31.227 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:18:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm_log_destination": { 2024-05-14T17:18:31.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm_log_destination", 2024-05-14T17:18:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "file", 2024-05-14T17:18:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "file,syslog", 2024-05-14T17:18:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: "syslog" 2024-05-14T17:18:31.232 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:18:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-05-14T17:18:31.234 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-05-14T17:18:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.236 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:18:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.238 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-05-14T17:18:31.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-05-14T17:18:31.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.239 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:18:31.240 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.240 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-05-14T17:18:31.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-05-14T17:18:31.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:18:31.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.244 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.245 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-05-14T17:18:31.246 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.247 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:18:31.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:18:31.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:18:31.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-05-14T17:18:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.259 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:18:31.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.269 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_samba": { 2024-05-14T17:18:31.269 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_samba", 2024-05-14T17:18:31.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.269 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:18:31.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.272 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.273 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.274 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.277 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-05-14T17:18:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-05-14T17:18:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.282 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.282 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-05-14T17:18:31.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-05-14T17:18:31.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.282 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:18:31.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.284 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-05-14T17:18:31.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-05-14T17:18:31.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:31.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.286 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.286 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:18:31.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.288 INFO:teuthology.orchestra.run.smithi099.stdout: "grafana_dashboards_path": { 2024-05-14T17:18:31.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:18:31.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:18:31.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.290 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:18:31.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.291 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-05-14T17:18:31.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-05-14T17:18:31.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.292 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:31.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:18:31.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.295 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-05-14T17:18:31.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-05-14T17:18:31.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.295 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.296 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.297 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-05-14T17:18:31.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-05-14T17:18:31.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.297 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:18:31.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.299 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.299 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.301 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.302 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.303 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.304 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.305 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.305 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.305 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.305 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.310 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.310 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.310 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.310 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.311 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.311 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.311 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.311 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.312 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:18:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.316 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.317 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-05-14T17:18:31.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-05-14T17:18:31.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.317 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-05-14T17:18:31.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-05-14T17:18:31.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.320 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.321 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.321 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.321 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-05-14T17:18:31.321 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.322 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-05-14T17:18:31.322 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-05-14T17:18:31.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.323 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-05-14T17:18:31.323 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.323 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.324 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-05-14T17:18:31.324 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-05-14T17:18:31.324 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:18:31.325 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.325 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.325 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-05-14T17:18:31.326 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-05-14T17:18:31.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-05-14T17:18:31.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.328 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:18:31.328 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.328 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.329 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-05-14T17:18:31.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:18:31.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.329 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:18:31.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.330 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.330 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.330 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:18:31.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-05-14T17:18:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-05-14T17:18:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.334 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-05-14T17:18:31.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:18:31.336 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.338 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.339 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.340 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.343 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:18:31.345 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.345 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.346 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.347 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:18:31.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.354 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.355 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.356 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.360 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.361 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.362 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.364 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.365 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.368 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:18:31.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:18:31.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.377 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:18:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.383 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.385 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:18:31.388 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.389 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:18:31.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:18:31.394 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:18:31.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.394 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.395 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.396 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:18:31.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:18:31.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.400 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.403 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.404 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.405 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.406 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "[]", 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "{}", 2024-05-14T17:18:31.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.409 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.410 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:18:31.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.412 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.415 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.416 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:18:31.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.425 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.431 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.432 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.433 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:18:31.433 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:18:31.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.434 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-05-14T17:18:31.434 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.434 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.434 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.434 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.435 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.436 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:18:31.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:18:31.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.437 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.437 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.437 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.439 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:18:31.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:18:31.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.440 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:31.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.441 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.441 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.442 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.442 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.442 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.443 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:18:31.443 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:18:31.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.443 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.445 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.446 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.446 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:18:31.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:18:31.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.447 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.448 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.448 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.448 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.450 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.450 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:18:31.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:18:31.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.452 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.452 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.453 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.453 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.454 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:18:31.454 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:18:31.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.455 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.455 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.456 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.456 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.456 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.457 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.457 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:18:31.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:18:31.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.458 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.458 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.458 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.458 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.459 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.459 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.459 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.460 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:18:31.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:18:31.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.460 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.461 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:31.461 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.461 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.461 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.461 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.461 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.464 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:18:31.466 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.468 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.471 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.472 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.473 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.474 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.474 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.479 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.479 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.479 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.479 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.481 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.482 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.483 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:31.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.492 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.495 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.498 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.502 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.503 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.507 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.508 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.509 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.510 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.512 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.514 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.515 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-05-14T17:18:31.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:18:31.517 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.518 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.519 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.520 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.521 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.523 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.525 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.528 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.531 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.533 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-05-14T17:18:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.537 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-05-14T17:18:31.539 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.540 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.541 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.542 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-05-14T17:18:31.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:18:31.544 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.545 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.547 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.548 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.551 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.552 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.553 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.554 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.558 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-05-14T17:18:31.560 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.564 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-05-14T17:18:31.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.567 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.568 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.570 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.571 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.572 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.573 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.574 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.576 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.579 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.580 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.583 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.586 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.587 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:18:31.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:18:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.594 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.595 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.596 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.597 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.598 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.599 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.601 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.603 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.606 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.607 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.608 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.610 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.611 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.612 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.618 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.619 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-05-14T17:18:31.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.624 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.626 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.627 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.628 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.632 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.634 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.635 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.637 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.638 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.639 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.641 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.642 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.643 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.644 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.646 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.647 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.648 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.650 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.651 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.652 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.653 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.655 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.656 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.657 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.658 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.659 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.661 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.663 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.664 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.666 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.667 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.668 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.669 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.671 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.672 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.674 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.675 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.677 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.678 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.679 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.681 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.682 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.685 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.686 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.687 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.689 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.691 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.693 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-05-14T17:18:31.694 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.695 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.696 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.698 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.700 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.701 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.702 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:31.703 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.704 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.705 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.708 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.710 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.711 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.713 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.715 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.716 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.717 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.718 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.719 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.720 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.721 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:31.722 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.723 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.724 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-05-14T17:18:31.725 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.727 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.728 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.729 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.730 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.731 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.731 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.731 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.731 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.731 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.733 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.734 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.735 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.736 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.737 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.738 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.739 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.740 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-05-14T17:18:31.741 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:31.742 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:18:31.743 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.744 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.745 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.746 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.748 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-05-14T17:18:31.749 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-05-14T17:18:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.754 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.755 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.756 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.758 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.759 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.760 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:18:31.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.762 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.763 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.764 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.765 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.766 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.768 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.769 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-05-14T17:18:31.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.772 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.773 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.775 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.777 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.778 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.779 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.782 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.785 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.786 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.787 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.789 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.790 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.791 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.794 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.797 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.798 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.799 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.800 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.801 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.803 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.804 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.805 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.806 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.809 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.810 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.811 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.812 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.813 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.814 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.815 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.816 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.818 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-05-14T17:18:31.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.820 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.821 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.823 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.824 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.825 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.826 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-05-14T17:18:31.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.829 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.831 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.834 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.835 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smb", 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.838 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.842 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.844 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.845 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "update_orchestration": { 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "update_orchestration", 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.847 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-05-14T17:18:31.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.851 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.852 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.853 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.854 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.855 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.856 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.857 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.858 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.859 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.861 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-05-14T17:18:31.862 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.863 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.864 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.867 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.868 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.869 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.870 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.871 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.872 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.873 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.874 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.875 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.877 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.878 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.879 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.882 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.884 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-05-14T17:18:31.886 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.887 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.888 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.889 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.891 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.892 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.893 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.894 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.896 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-05-14T17:18:31.897 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.898 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.899 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.900 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.902 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.903 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-05-14T17:18:31.904 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:18:31.905 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.906 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.907 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.908 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-05-14T17:18:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.911 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.912 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.916 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-05-14T17:18:31.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.918 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.920 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.921 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.922 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.925 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.926 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.929 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.930 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.932 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.935 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.937 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.938 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.939 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.939 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.939 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.939 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.942 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.942 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.942 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.942 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.942 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.942 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.945 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.948 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.949 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.950 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.951 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.952 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.953 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.954 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.955 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "periodic_async_work": { 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "periodic_async_work", 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.957 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Periodically check for async work", 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.959 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.960 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.961 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.962 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:31.964 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.965 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.966 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:31.967 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.968 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.970 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.971 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.972 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.973 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.974 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.976 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:31.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-05-14T17:18:31.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.984 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 24811, 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "z", 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540701547738038271, 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:31.985 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.986 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.988 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.989 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.990 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.991 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:31.992 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.994 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-05-14T17:18:31.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.997 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:31.999 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.000 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.001 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.002 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-05-14T17:18:32.003 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:18:32.004 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.005 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.007 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-05-14T17:18:32.010 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.011 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.013 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.014 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.016 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.017 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-05-14T17:18:32.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:18:32.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.020 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-05-14T17:18:32.022 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.023 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.024 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.026 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.027 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.028 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.029 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.030 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.031 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.032 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-05-14T17:18:32.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "read", 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap", 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap-read" 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.034 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.035 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.036 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.037 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.038 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:32.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.041 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.042 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.043 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.046 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-05-14T17:18:32.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.049 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.050 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.051 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.052 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm_log_destination": { 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm_log_destination", 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.054 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "file", 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "file,syslog", 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "syslog" 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.056 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-05-14T17:18:32.057 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.058 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.061 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-05-14T17:18:32.062 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:18:32.063 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:18:32.064 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-05-14T17:18:32.065 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.066 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.069 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:18:32.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.072 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.073 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.074 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.076 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.077 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-05-14T17:18:32.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-05-14T17:18:32.079 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.080 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.082 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_samba": { 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_samba", 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.083 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.084 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:18:32.085 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.086 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.087 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.091 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.092 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.093 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-05-14T17:18:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-05-14T17:18:32.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.096 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-05-14T17:18:32.097 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.098 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "grafana_dashboards_path": { 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-05-14T17:18:32.102 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:18:32.103 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.104 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.105 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.106 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.107 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.108 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.110 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.111 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.112 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.114 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:18:32.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-05-14T17:18:32.117 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.118 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.119 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.121 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.122 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.123 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-05-14T17:18:32.124 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-05-14T17:18:32.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:18:32.126 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.127 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-05-14T17:18:32.128 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.129 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.132 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.133 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.134 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.135 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-05-14T17:18:32.136 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:18:32.137 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.139 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.140 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.141 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.142 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-05-14T17:18:32.144 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:18:32.145 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.146 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.147 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.148 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-05-14T17:18:32.149 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.150 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.154 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.155 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.157 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.158 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.159 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.160 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.163 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.164 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.165 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.166 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.167 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:18:32.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.169 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.172 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.175 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.175 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:18:32.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:18:32.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.175 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.177 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:18:32.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.180 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:18:32.181 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.182 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.183 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.185 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.186 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.187 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.188 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:18:32.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:32.191 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:18:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "[]", 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:18:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "{}", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:18:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:18:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:18:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:18:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:18:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-05-14T17:18:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-05-14T17:18:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:18:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-05-14T17:18:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-05-14T17:18:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-05-14T17:18:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:18:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-05-14T17:18:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-05-14T17:18:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:18:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-05-14T17:18:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-05-14T17:18:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-05-14T17:18:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.432 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.435 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.436 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.437 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.440 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.441 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.442 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.443 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.445 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.447 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.448 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.451 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.452 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.455 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.457 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.458 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.459 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.461 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.464 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.469 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.470 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.471 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.474 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.475 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.478 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.479 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:32.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:18:32.483 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-05-14T17:18:32.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.497 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.498 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:18:32.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.500 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.503 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.504 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.506 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-05-14T17:18:32.507 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.510 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.511 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.514 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.515 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.516 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.518 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.520 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.523 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.524 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.525 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.526 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-05-14T17:18:32.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.531 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.532 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.533 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.534 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.535 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.536 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:32.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.543 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.544 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.546 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.548 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.550 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.551 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.552 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.553 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.554 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.555 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.556 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.558 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.559 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.562 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.563 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.564 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.567 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.568 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.570 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.571 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.572 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.573 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.574 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-05-14T17:18:32.575 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.578 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.579 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-05-14T17:18:32.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.581 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.582 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.583 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.584 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.585 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.587 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-05-14T17:18:32.588 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.591 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.592 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.594 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.595 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smb", 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.598 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.600 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.602 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.603 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.604 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "update_orchestration": { 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "update_orchestration", 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.607 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.609 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.610 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.611 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.613 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.615 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.617 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.619 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.620 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.621 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.622 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.623 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.624 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.625 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.627 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.629 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.630 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.631 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-05-14T17:18:32.632 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.634 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.637 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.638 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.639 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.641 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-05-14T17:18:32.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.644 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-05-14T17:18:32.645 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.647 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.649 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-05-14T17:18:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.665 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-05-14T17:18:32.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.667 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.668 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.669 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.670 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.671 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.672 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.673 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-05-14T17:18:32.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.675 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.678 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.679 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.682 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.683 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.685 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.686 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-05-14T17:18:32.687 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.688 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.691 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.693 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.694 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.696 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.697 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.698 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.698 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.698 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.698 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.698 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.702 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.702 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.702 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.702 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.702 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.702 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.705 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.708 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.713 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.714 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-05-14T17:18:32.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "periodic_async_work": { 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "periodic_async_work", 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Periodically check for async work", 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-05-14T17:18:32.718 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.719 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.722 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.726 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.727 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.728 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.729 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.730 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.733 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.734 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.735 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.736 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-05-14T17:18:32.738 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.742 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "modules": [ 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard", 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "iostat", 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "nfs", 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "restful", 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_schedule" 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-05-14T17:18:32.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.746 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.747 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.748 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.751 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.753 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.755 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.760 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-05-14T17:18:32.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.762 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.762 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-05-14T17:18:32.762 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.762 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.762 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.763 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-05-14T17:18:32.763 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.763 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.763 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-05-14T17:18:32.764 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-05-14T17:18:32.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.764 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.764 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.764 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:18:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.766 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-05-14T17:18:32.766 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-05-14T17:18:32.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.766 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.767 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.767 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.767 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.767 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.767 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:18:32.767 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.768 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.768 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-05-14T17:18:32.768 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-05-14T17:18:32.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.769 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.769 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.769 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.769 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.769 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.770 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-05-14T17:18:32.770 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.770 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.770 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.771 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.772 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.773 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-05-14T17:18:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-05-14T17:18:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:18:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-05-14T17:18:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-05-14T17:18:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: "read", 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap", 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap-read" 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-05-14T17:18:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:18:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.806 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:18:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.808 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.809 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.810 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.811 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.812 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:18:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:18:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm_log_destination": { 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm_log_destination", 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "file", 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "file,syslog", 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "syslog" 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:18:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:18:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-05-14T17:18:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:18:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.826 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-05-14T17:18:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:18:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-05-14T17:18:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:18:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-05-14T17:18:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-05-14T17:18:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-05-14T17:18:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_samba": { 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_samba", 2024-05-14T17:18:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:18:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:18:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-05-14T17:18:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-05-14T17:18:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-05-14T17:18:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:18:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "grafana_dashboards_path": { 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:18:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.872 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:18:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-05-14T17:18:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-05-14T17:18:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-05-14T17:18:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-05-14T17:18:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:18:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-05-14T17:18:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-05-14T17:18:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-05-14T17:18:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:18:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-05-14T17:18:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:18:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.932 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.933 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.934 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:18:32.935 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.936 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.937 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.938 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.939 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.940 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.941 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.942 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:18:32.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:18:32.946 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.947 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.948 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.949 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.950 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.951 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.952 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.953 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:18:32.954 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.955 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.956 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.958 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:18:32.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.961 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:18:32.962 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "[]", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.963 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "{}", 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.967 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.969 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:18:32.970 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.971 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.972 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.973 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.974 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.976 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.977 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:18:32.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.980 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.981 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.982 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.985 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:18:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.987 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.988 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.990 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:18:32.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:18:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-05-14T17:18:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-05-14T17:18:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:18:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-05-14T17:18:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-05-14T17:18:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-05-14T17:18:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-05-14T17:18:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-05-14T17:18:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:18:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-05-14T17:18:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:18:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.094 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-05-14T17:18:33.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.098 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.099 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.100 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.103 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.104 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.106 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.109 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.109 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.110 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.111 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.112 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.113 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.116 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.117 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.120 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:18:33.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:33.124 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.125 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.126 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.127 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.128 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.129 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.130 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.132 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.133 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.134 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-05-14T17:18:33.135 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-05-14T17:18:33.136 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.137 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.138 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.140 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.141 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.142 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.143 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.144 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-05-14T17:18:33.145 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.146 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.147 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.148 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.149 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-05-14T17:18:33.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:18:33.151 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.152 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.153 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.154 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.155 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.157 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.158 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.159 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.160 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.161 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.162 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.163 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.164 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.165 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.166 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.167 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.168 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.169 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.170 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.171 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.173 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.174 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.175 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-05-14T17:18:33.176 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.177 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.178 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.180 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.181 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.182 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.183 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.184 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.185 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-05-14T17:18:33.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:18:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.189 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.190 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.192 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.193 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.194 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-05-14T17:18:33.198 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.199 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.201 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.202 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.204 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.206 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.208 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.211 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-05-14T17:18:33.221 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.222 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.223 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.226 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.227 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.230 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.231 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-05-14T17:18:33.235 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.236 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.240 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.241 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.247 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.256 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.258 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.260 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.263 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:33.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.271 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.276 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-05-14T17:18:33.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.279 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.280 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.282 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.284 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.287 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.288 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.298 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.301 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.302 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.305 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.309 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-05-14T17:18:33.310 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.313 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.313 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.313 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.314 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.318 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.319 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.322 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.323 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.325 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.329 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.333 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.335 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.336 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.339 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.340 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-05-14T17:18:33.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.344 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.345 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.346 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-05-14T17:18:33.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.353 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-05-14T17:18:33.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-05-14T17:18:33.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.360 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smb", 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.366 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.368 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.370 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "update_orchestration": { 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "update_orchestration", 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.381 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.385 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.390 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.391 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.392 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.393 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.394 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.396 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.398 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-05-14T17:18:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:18:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-05-14T17:18:33.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-05-14T17:18:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "periodic_async_work": { 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "periodic_async_work", 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Periodically check for async work", 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-05-14T17:18:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:18:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-05-14T17:18:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-05-14T17:18:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-05-14T17:18:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-05-14T17:18:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "sqlite3_killpoint": { 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "dev", 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-05-14T17:18:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-05-14T17:18:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-05-14T17:18:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "services": { 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard": "https://172.21.15.99:8443/" 2024-05-14T17:18:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "always_on_modules": { 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "octopus": [ 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-05-14T17:18:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "pacific": [ 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-05-14T17:18:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "quincy": [ 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-05-14T17:18:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "reef": [ 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-05-14T17:18:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "squid": [ 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-05-14T17:18:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-05-14T17:18:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "last_failure_osd_epoch": 83, 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "active_clients": [ 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3369633141 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "libcephsqlite", 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3571059176 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-05-14T17:18:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-05-14T17:18:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 210173666 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4132443911 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 679199697 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-05-14T17:18:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3056287268 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-14T17:18:33.525 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:33.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:31 smithi193 ceph-mon[35962]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:18:33.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:31 smithi193 ceph-mon[35962]: pgmap v469: 97 pgs: 97 active+clean; 401 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 19 MiB/s wr, 276 op/s 2024-05-14T17:18:33.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:31 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/393965112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:18:33.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:31 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4177822463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:18:33.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:31 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/4182272714' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:32 smithi193 ceph-mon[35962]: pgmap v470: 97 pgs: 97 active+clean; 401 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 18 MiB/s wr, 219 op/s 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:31 smithi150 ceph-mon[36204]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:31 smithi150 ceph-mon[36204]: pgmap v469: 97 pgs: 97 active+clean; 401 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 19 MiB/s wr, 276 op/s 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:31 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/393965112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:31 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4177822463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:31 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/4182272714' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:18:33.528 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:32 smithi150 ceph-mon[36204]: pgmap v470: 97 pgs: 97 active+clean; 401 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 18 MiB/s wr, 219 op/s 2024-05-14T17:18:33.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81f80b29e0 msgr2=0x7f81f80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81f80b29e0 0x7f81f80b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f820c06a730 tx=0x7f81fc005e70 comp rx=0 tx=0).stop 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 msgr2=0x7f820c19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 0x7f820c19a050 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8208004770 tx=0x7f82080042f0 comp rx=0 tx=0).stop 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 shutdown_connections 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f81f80b29e0 0x7f81f80b4ea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c107940 0x7f820c1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f820c1022f0 0x7f820c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 --2- 172.21.15.99:0/4182272714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f820c101930 0x7f820c19a050 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 >> 172.21.15.99:0/4182272714 conn(0x7f820c078a50 msgr2=0x7f820c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 shutdown_connections 2024-05-14T17:18:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:18:31.130+0000 7f8213c50640 1 -- 172.21.15.99:0/4182272714 wait complete. 2024-05-14T17:18:33.539 DEBUG:tasks.check_counter:Getting stats from a 2024-05-14T17:18:33.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-14T17:18:33.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:31 smithi099 ceph-mon[31851]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:18:33.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:31 smithi099 ceph-mon[31851]: pgmap v469: 97 pgs: 97 active+clean; 401 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 19 MiB/s wr, 276 op/s 2024-05-14T17:18:33.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:31 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/393965112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:18:33.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:31 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4177822463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:18:33.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:31 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/4182272714' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:18:33.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:32 smithi099 ceph-mon[31851]: pgmap v470: 97 pgs: 97 active+clean; 401 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 18 MiB/s wr, 219 op/s 2024-05-14T17:18:33.794 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:35.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:34 smithi150 ceph-mon[36204]: pgmap v471: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 20 MiB/s wr, 254 op/s 2024-05-14T17:18:35.204 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:34 smithi099 ceph-mon[31851]: pgmap v471: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 20 MiB/s wr, 254 op/s 2024-05-14T17:18:35.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:34 smithi193 ceph-mon[35962]: pgmap v471: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 20 MiB/s wr, 254 op/s 2024-05-14T17:18:35.642 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-05-14T17:18:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.001214172, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000233340, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 6, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 231, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3312, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 198144, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.017784762, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.012658522, 2024-05-14T17:18:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.002251029, 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 229, 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.018331972, 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000080052 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 198144 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 311, 2024-05-14T17:18:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 300, 2024-05-14T17:18:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1736464, 2024-05-14T17:18:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 184880, 2024-05-14T17:18:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-05-14T17:18:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-05-14T17:18:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.050156835, 2024-05-14T17:18:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.016735209, 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.037280545, 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.005484056, 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 299, 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.012360692, 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000041340 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1736464, 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 184880 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-14T17:18:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-14T17:18:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148771, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 32, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-14T17:18:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 313, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1692544, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 313, 2024-05-14T17:18:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 313, 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1692544, 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:35.662 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:36.423 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-14T17:18:36.424 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-14T17:18:36.424 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-14T17:18:36.424 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-14T17:18:36.425 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-14T17:18:36.425 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-14T17:18:36.425 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-14T17:18:36.425 DEBUG:tasks.check_counter:Getting stats from d 2024-05-14T17:18:36.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-14T17:18:36.667 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:37.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:36 smithi150 ceph-mon[36204]: pgmap v472: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-14T17:18:37.246 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:36 smithi099 ceph-mon[31851]: pgmap v472: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-14T17:18:37.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:36 smithi193 ceph-mon[35962]: pgmap v472: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-14T17:18:38.390 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 4, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 4, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 2992, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 448, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.001477364, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000033672, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000563022, 2024-05-14T17:18:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000762074, 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000254024 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2, 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 233, 2024-05-14T17:18:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 320, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 198400, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.018249104, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.013462969, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001744031, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 230, 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.015038050, 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000065382 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 198400 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 317, 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 306, 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1738096, 2024-05-14T17:18:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 188720, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.054719840, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.018511468, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.040072579, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.005702338, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 305, 2024-05-14T17:18:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.013117231, 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000043007 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1738096, 2024-05-14T17:18:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 188720 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148692, 2024-05-14T17:18:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 30, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 319, 2024-05-14T17:18:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1693324, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 319, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 319, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1693324, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:38.408 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:39.001 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-14T17:18:39.002 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-14T17:18:39.002 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-14T17:18:39.002 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-14T17:18:39.002 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-14T17:18:39.002 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-14T17:18:39.002 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-14T17:18:39.002 DEBUG:tasks.check_counter:Getting stats from g 2024-05-14T17:18:39.003 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-14T17:18:39.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:38 smithi150 ceph-mon[36204]: pgmap v473: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-14T17:18:39.245 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:39.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:38 smithi099 ceph-mon[31851]: pgmap v473: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-14T17:18:39.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:38 smithi193 ceph-mon[35962]: pgmap v473: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 2.4 MiB/s wr, 97 op/s 2024-05-14T17:18:40.855 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-05-14T17:18:40.856 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.001172786, 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000253938, 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:40.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:18:40.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 232, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 277184, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.018765060, 2024-05-14T17:18:40.859 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.013628352, 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.002048814, 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 229, 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.017498105, 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000076410 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.860 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 277184 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:40.861 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 317, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 304, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1740000, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 186608, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.051286799, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.017966012, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.037192516, 2024-05-14T17:18:40.862 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.005368986, 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 304, 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.013226060, 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000043506 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1740000, 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 186608 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-14T17:18:40.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:40.865 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:40.867 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:40.868 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 147908, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 28, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-14T17:18:40.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:40.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:40.871 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-14T17:18:40.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:40.872 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:40.872 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 316, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1692831, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 316, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-14T17:18:40.873 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 316, 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1692831, 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:40.874 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:41.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:40 smithi150 ceph-mon[36204]: pgmap v474: 97 pgs: 97 active+clean; 398 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-05-14T17:18:41.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:40 smithi150 ceph-mon[36204]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:41.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:40 smithi099 ceph-mon[31851]: pgmap v474: 97 pgs: 97 active+clean; 398 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-05-14T17:18:41.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:40 smithi099 ceph-mon[31851]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:40 smithi193 ceph-mon[35962]: pgmap v474: 97 pgs: 97 active+clean; 398 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 4.1 MiB/s wr, 161 op/s 2024-05-14T17:18:41.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:40 smithi193 ceph-mon[35962]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:18:41.491 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-14T17:18:41.491 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-14T17:18:41.491 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-14T17:18:41.491 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-14T17:18:41.492 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-14T17:18:41.492 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-14T17:18:41.492 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-14T17:18:41.492 DEBUG:tasks.check_counter:Getting stats from j 2024-05-14T17:18:41.492 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-14T17:18:41.733 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:43.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:42 smithi099 ceph-mon[31851]: pgmap v475: 97 pgs: 97 active+clean; 398 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-05-14T17:18:43.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:42 smithi150 ceph-mon[36204]: pgmap v475: 97 pgs: 97 active+clean; 398 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-05-14T17:18:43.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:42 smithi193 ceph-mon[35962]: pgmap v475: 97 pgs: 97 active+clean; 398 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-05-14T17:18:43.411 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-14T17:18:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.001034010, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000181415, 2024-05-14T17:18:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2024-05-14T17:18:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 236, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 277888, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.021213037, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.015991521, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.002029916, 2024-05-14T17:18:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 233, 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.017564991, 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000075386 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 277888 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 323, 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 310, 2024-05-14T17:18:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1741632, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 190448, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.055160189, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.019773080, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.039755974, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.005666296, 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 310, 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.015312142, 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000049394 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1741632, 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 190448 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-14T17:18:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-14T17:18:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 147930, 2024-05-14T17:18:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 28, 2024-05-14T17:18:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-14T17:18:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 322, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1693611, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 322, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 322, 2024-05-14T17:18:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1693611, 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-14T17:18:43.430 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-14T17:18:44.140 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-14T17:18:44.140 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-14T17:18:44.140 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-14T17:18:44.140 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-14T17:18:44.141 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-14T17:18:44.141 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-14T17:18:44.141 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-14T17:18:44.141 DEBUG:tasks.check_counter:Getting stats from b 2024-05-14T17:18:44.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-14T17:18:44.390 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:18:45.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:44 smithi150 ceph-mon[36204]: pgmap v476: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 3.6 MiB/s wr, 138 op/s 2024-05-14T17:18:45.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:44 smithi099 ceph-mon[31851]: pgmap v476: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 3.6 MiB/s wr, 138 op/s 2024-05-14T17:18:45.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:44 smithi193 ceph-mon[35962]: pgmap v476: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 3.6 MiB/s wr, 138 op/s 2024-05-14T17:18:46.149 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-14T17:18:46.149 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:46.149 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 173279, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 171654, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 53122664, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 112428139, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 10, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 7.977623967, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 3.302638947, 2024-05-14T17:18:46.150 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 5.931091127, 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.169177257, 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 171654, 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.464256412, 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000014355 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:46.151 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 321732, 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.038747533, 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000120 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:46.152 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 12103, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 14125, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3333690, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 40332087, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 13, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 8, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.807451811, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.294597953, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.270305484, 2024-05-14T17:18:46.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.208850978, 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 14123, 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.344860413, 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000024418 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8875, 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001246422, 2024-05-14T17:18:46.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000140 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 155824 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3825, 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 3997, 2024-05-14T17:18:46.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3290814, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 6608817, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 14, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.311982533, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.114406956, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.103564818, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.085255499, 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:46.156 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3994, 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.143440102, 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000035913 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 50, 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000008811, 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000176 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1878080, 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 706944 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-05-14T17:18:46.158 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 22572, 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 11.994611155, 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000531393 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-14T17:18:46.159 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 9943, 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.615986998, 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000061951 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 45486, 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 35807, 2024-05-14T17:18:46.160 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 35807, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 309.959488996, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.008656393 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 9679, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 3, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-05-14T17:18:46.161 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 11, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 1039, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 110, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 829, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 100, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 829, 2024-05-14T17:18:46.162 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 326, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 61, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 61, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 20, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 90769, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 43630, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 9678, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 188, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:18:46.163 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 1875, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 14573, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 3, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 203, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 17, 2024-05-14T17:18:46.164 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 943, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 766, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 4992489, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-05-14T17:18:46.165 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 1993, 2024-05-14T17:18:46.166 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 23928, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 12867, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 7164, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 2703, 2024-05-14T17:18:46.167 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 34752 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 265, 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 514, 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 312, 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 483, 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 303, 2024-05-14T17:18:46.168 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 2, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 5512, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 5510, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:18:46.169 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:18:46.170 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-05-14T17:18:46.170 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-05-14T17:18:46.170 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 12, 2024-05-14T17:18:46.170 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 560, 2024-05-14T17:18:46.170 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:18:46.170 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 5, 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-05-14T17:18:46.171 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 40154, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 40154, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 50, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-05-14T17:18:46.172 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 50, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 5, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 72730073, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-05-14T17:18:46.173 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 11398, 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 21.544982536, 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001890242 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-05-14T17:18:46.174 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 832, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 6571, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 5739, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 148, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 249, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 101, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 1078, 2024-05-14T17:18:46.175 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 9013, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 7935, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 59, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 5736, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 5677, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 69384, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 223608 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.176 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 74425, 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 14, 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 45486, 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 1372, 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 8371, 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5439, 2024-05-14T17:18:46.177 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 10.341612045, 2024-05-14T17:18:46.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001901381 2024-05-14T17:18:46.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.178 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-05-14T17:18:46.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 92, 2024-05-14T17:18:46.178 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.065420862, 2024-05-14T17:18:46.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000711096 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.179 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 9939, 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 6.898666262, 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000694100 2024-05-14T17:18:46.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000143361, 2024-05-14T17:18:46.181 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000143361 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.182 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.184 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-05-14T17:18:46.184 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.184 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.184 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.184 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.185 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-05-14T17:18:46.186 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 12868, 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 5.750074128, 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000446850 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.187 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1932, 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 190.917471340, 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.098818566 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.189 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.190 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 24, 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.095806558, 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003991939 2024-05-14T17:18:46.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.192 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.193 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-05-14T17:18:46.194 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5512, 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 95.890294440, 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.017396642 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.195 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 3, 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 1, 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-05-14T17:18:46.196 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 3235, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 1617, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 875, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:46.197 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:46.198 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:46.199 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:46.200 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:46.201 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 413623, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 902, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:46.202 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8040, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 168, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:46.203 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 2953377, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 26563, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.204 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 18438, 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 69681724, 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 30, 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 18438, 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-05-14T17:18:46.205 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 18438, 2024-05-14T17:18:46.206 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 46.269156115, 2024-05-14T17:18:46.206 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002509445 2024-05-14T17:18:46.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.206 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-05-14T17:18:46.206 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-05-14T17:18:46.206 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 18438, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 19031 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 18438, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 420, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 18018, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-05-14T17:18:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 417, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 11815, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 252, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-05-14T17:18:46.208 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 5538, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2024-05-14T17:18:46.209 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:18:46.210 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-05-14T17:18:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-05-14T17:18:46.212 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 103, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 23, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-05-14T17:18:46.213 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 722, 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 6, 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 250 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:18:46.214 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 431 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 10, 2024-05-14T17:18:46.215 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 10, 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 5510, 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 5510, 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:46.216 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 189203, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 42695285, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 189203, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:46.217 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 189203, 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 42695285, 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.218 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 18438, 2024-05-14T17:18:46.219 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 70240282, 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 18438, 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 70240282, 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.220 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:46.221 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 18438, 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 18438, 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 18438, 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 18438, 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 5510, 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 558606, 2024-05-14T17:18:46.223 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 5510, 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 417, 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 558606, 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:46.224 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55bf9d602500": { 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 17169, 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:46.225 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 40154, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 68535769, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 40154, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 11398, 2024-05-14T17:18:46.226 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 68518600, 2024-05-14T17:18:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:46.228 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds.imported=17 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=265 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=514 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:18:47.030 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:18:47.031 DEBUG:tasks.check_counter:Getting stats from e 2024-05-14T17:18:47.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-14T17:18:47.055 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:46 smithi150 ceph-mon[36204]: pgmap v477: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-05-14T17:18:47.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:46 smithi099 ceph-mon[31851]: pgmap v477: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-05-14T17:18:47.281 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:18:47.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:46 smithi193 ceph-mon[35962]: pgmap v477: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-05-14T17:18:48.864 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-14T17:18:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 319, 2024-05-14T17:18:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 308, 2024-05-14T17:18:48.864 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1738640, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 190000, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.059273201, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.019160292, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.041413562, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.007145795, 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:48.865 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 307, 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.016087382, 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000052401 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:48.866 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1738640, 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 190000 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 7, 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 239, 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3472, 2024-05-14T17:18:48.867 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 278512, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.015569400, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.010437567, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.002221615, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 237, 2024-05-14T17:18:48.868 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.023589295, 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000099532 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-05-14T17:18:48.869 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 278512 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-14T17:18:48.870 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000439019, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000376779, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:48.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:18:48.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:48.873 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:48.874 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:48.875 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:48.876 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148000, 2024-05-14T17:18:48.877 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 29, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:48.878 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:48.879 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:48.880 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 322, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1693611, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 322, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 322, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1693611, 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:48.881 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:48.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:48.882 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-14T17:18:49.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:48 smithi150 ceph-mon[36204]: pgmap v478: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-05-14T17:18:49.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:48 smithi099 ceph-mon[31851]: pgmap v478: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-05-14T17:18:49.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:48 smithi193 ceph-mon[35962]: pgmap v478: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-05-14T17:18:49.587 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-05-14T17:18:49.588 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-05-14T17:18:49.588 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-05-14T17:18:49.588 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-05-14T17:18:49.588 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-05-14T17:18:49.588 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-05-14T17:18:49.588 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-05-14T17:18:49.588 DEBUG:tasks.check_counter:Getting stats from h 2024-05-14T17:18:49.589 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-14T17:18:49.867 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:18:51.050 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:50 smithi150 ceph-mon[36204]: pgmap v479: 97 pgs: 97 active+clean; 412 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 3.4 MiB/s wr, 131 op/s 2024-05-14T17:18:51.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:50 smithi099 ceph-mon[31851]: pgmap v479: 97 pgs: 97 active+clean; 412 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 3.4 MiB/s wr, 131 op/s 2024-05-14T17:18:51.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:50 smithi193 ceph-mon[35962]: pgmap v479: 97 pgs: 97 active+clean; 412 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 3.4 MiB/s wr, 131 op/s 2024-05-14T17:18:51.609 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-14T17:18:51.609 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:51.609 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 957, 2024-05-14T17:18:51.609 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 957, 2024-05-14T17:18:51.609 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 21220325, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 303566, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 8, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.094298899, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.027805279, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.034748610, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.023618772, 2024-05-14T17:18:51.610 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 956, 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.033685302, 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000035235 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1138, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1287, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 28659089, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 854821, 2024-05-14T17:18:51.612 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.149189516, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.042567222, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.080339153, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.029068297, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1286, 2024-05-14T17:18:51.613 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.060974159, 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000047413 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.614 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1865104, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 610592 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1783, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1832, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 21513496, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 721364, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2024-05-14T17:18:51.615 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.195789095, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.055518752, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.061380877, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.061700983, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1829, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.077263105, 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000042243 2024-05-14T17:18:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 154688 2024-05-14T17:18:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-14T17:18:51.618 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 6020, 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.160602728, 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000026678 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-05-14T17:18:51.621 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 6508, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 105, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 6403, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 6359, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-05-14T17:18:51.622 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 16, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-05-14T17:18:51.623 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:18:51.624 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:18:51.625 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:18:51.626 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-05-14T17:18:51.627 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-05-14T17:18:51.628 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:18:51.629 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-05-14T17:18:51.630 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 40155, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-05-14T17:18:51.631 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 50, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 5, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 72736891, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 72730073, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.632 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 40155 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 754, 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 6451, 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 5697, 2024-05-14T17:18:51.633 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 59, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 62, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 3, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 6508, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 6715, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 207, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-05-14T17:18:51.634 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 60484, 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215416 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-05-14T17:18:51.635 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.636 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.637 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.638 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-05-14T17:18:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-05-14T17:18:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-05-14T17:18:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-05-14T17:18:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-05-14T17:18:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 3, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 1, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 3, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-05-14T17:18:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 3093, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 244839, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 830, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8040, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 168, 2024-05-14T17:18:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 6491293, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 43276, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-14T17:18:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 3503, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-05-14T17:18:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 3503, 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3503, 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.159960673, 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000616603 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-05-14T17:18:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3503, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 3504 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 3503, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 3503, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 2008, 2024-05-14T17:18:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 1493, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-05-14T17:18:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:18:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-05-14T17:18:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-05-14T17:18:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 103, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 23, 2024-05-14T17:18:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 8, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 9, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 1, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-05-14T17:18:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 3874, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 71022448, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 3874, 2024-05-14T17:18:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3874, 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 71022448, 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 3503, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 68543734, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3503, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 68543734, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-14T17:18:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 3503, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 3503, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3503, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 3503, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-14T17:18:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55a73b4286e0": { 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-14T17:18:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:51.684 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-14T17:18:52.396 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-14T17:18:52.397 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-14T17:18:52.397 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-14T17:18:52.397 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-14T17:18:52.397 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-05-14T17:18:52.397 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:18:52.397 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:18:52.397 DEBUG:tasks.check_counter:Getting stats from k 2024-05-14T17:18:52.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-14T17:18:52.636 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:18:53.009 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:52 smithi150 ceph-mon[36204]: pgmap v480: 97 pgs: 97 active+clean; 412 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 1.8 MiB/s wr, 66 op/s 2024-05-14T17:18:53.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:52 smithi099 ceph-mon[31851]: pgmap v480: 97 pgs: 97 active+clean; 412 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 1.8 MiB/s wr, 66 op/s 2024-05-14T17:18:53.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:52 smithi193 ceph-mon[35962]: pgmap v480: 97 pgs: 97 active+clean; 412 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 1.8 MiB/s wr, 66 op/s 2024-05-14T17:18:54.308 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-14T17:18:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-14T17:18:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-14T17:18:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:18:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-14T17:18:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001129085, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000278047, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:54.310 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:18:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 327, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 314, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1742720, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 193008, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.064605895, 2024-05-14T17:18:54.312 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.019497493, 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.046357749, 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.007793436, 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 314, 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.016538385, 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000052670 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.313 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1742720, 2024-05-14T17:18:54.314 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 193008 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 240, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 278592, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-14T17:18:54.315 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.016268619, 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.011197828, 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001766833, 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 237, 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.021146970, 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000089227 2024-05-14T17:18:54.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 278592 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.317 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:54.318 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:54.319 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:54.320 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:54.321 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:54.322 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 147937, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 28, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:54.323 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-14T17:18:54.324 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-14T17:18:54.325 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 326, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1694131, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 326, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-14T17:18:54.326 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 326, 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1694131, 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-14T17:18:54.327 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-14T17:18:54.954 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-14T17:18:54.954 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-14T17:18:54.954 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-14T17:18:54.954 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-14T17:18:54.955 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-14T17:18:54.955 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-14T17:18:54.955 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-14T17:18:54.955 DEBUG:tasks.check_counter:Getting stats from c 2024-05-14T17:18:54.955 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-14T17:18:55.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:54 smithi150 ceph-mon[36204]: pgmap v481: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-05-14T17:18:55.215 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:18:55.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:54 smithi099 ceph-mon[31851]: pgmap v481: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-05-14T17:18:55.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:54 smithi193 ceph-mon[35962]: pgmap v481: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.9 MiB/s wr, 93 op/s 2024-05-14T17:18:56.670 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T17:18:56.670 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:56.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4544, 2024-05-14T17:18:56.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 4570, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2919944, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 22615790, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 20, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 10, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.363072333, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.141516312, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.109862748, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.098272121, 2024-05-14T17:18:56.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4564, 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.116431428, 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000025510 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 112, 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000038227, 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000341 2024-05-14T17:18:56.672 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1888864, 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 223184 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 106006, 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 64036, 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 32828061, 2024-05-14T17:18:56.673 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 22804777, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 4.777887415, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 2.097993887, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 3.627041558, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.052443543, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 64039, 2024-05-14T17:18:56.674 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.928908232, 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000014505 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 165749, 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.020868005, 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000125 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 5792, 2024-05-14T17:18:56.675 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 161072 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 60241, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 74467, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 19001628, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 91892041, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-05-14T17:18:56.676 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 2.836103399, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 1.223185030, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 1.890363130, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.091445478, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 74465, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.542664315, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000020716 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.677 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 119504, 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.016293478, 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000136 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 418144 2024-05-14T17:18:56.678 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:18:56.679 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 11204, 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 7.100008599, 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000633703 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:18:56.680 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4438, 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.232416578, 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000052369 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T17:18:56.681 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 40414, 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 29862, 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 29862, 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 79.302972247, 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002655648 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T17:18:56.682 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 10552, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 5, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 19, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 1, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 1078, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 98, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 647, 2024-05-14T17:18:56.683 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 333, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 938, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 559, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 53, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 54, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 19, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 57726, 2024-05-14T17:18:56.684 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 27269, 2024-05-14T17:18:56.685 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 10545, 2024-05-14T17:18:56.685 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 138, 2024-05-14T17:18:56.685 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:18:56.685 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T17:18:56.685 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 75, 2024-05-14T17:18:56.685 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 18485, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 12, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 418, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 9, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 927, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:18:56.686 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 766, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 4992489, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:18:56.687 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 3, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 3865, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 15404, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:18:56.688 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 11792, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 3904, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 2274, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 28100 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 90, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 853, 2024-05-14T17:18:56.689 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 479, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 796, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 149, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 2297, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 2297, 2024-05-14T17:18:56.690 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 12, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 863, 2024-05-14T17:18:56.691 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 32, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 1, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T17:18:56.692 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 31435, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 31435, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 38, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-05-14T17:18:56.693 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 38, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 4, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 49726974, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-05-14T17:18:56.694 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5996, 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 18.825367038, 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003139654 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T17:18:56.695 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 716, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 3494, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 2778, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 110, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 450, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 340, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 1074, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 6651, 2024-05-14T17:18:56.696 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 5577, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 54, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 4073, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 4019, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 70928, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223608 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 52498, 2024-05-14T17:18:56.697 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 831, 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 40414, 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 431, 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 3928, 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2353, 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 27.399779649, 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.011644615 2024-05-14T17:18:56.698 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 12, 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.024231847, 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002019320 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4196, 2024-05-14T17:18:56.700 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 11.240301855, 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002678813 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.701 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000143699, 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000143699 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.703 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.703 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T17:18:56.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.703 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.704 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T17:18:56.704 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.705 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.707 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T17:18:56.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.708 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.708 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T17:18:56.708 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.709 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.709 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.709 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.709 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T17:18:56.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.710 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.710 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.710 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.711 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T17:18:56.711 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.711 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.712 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.712 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T17:18:56.712 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.713 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.713 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.713 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T17:18:56.713 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 20929, 2024-05-14T17:18:56.714 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 9.453757778, 2024-05-14T17:18:56.714 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000451706 2024-05-14T17:18:56.714 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.715 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T17:18:56.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 74, 2024-05-14T17:18:56.715 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.146943500, 2024-05-14T17:18:56.715 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001985722 2024-05-14T17:18:56.716 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.716 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T17:18:56.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.716 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.716 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.717 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.717 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T17:18:56.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.717 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.718 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.718 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.718 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T17:18:56.719 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.719 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.719 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.719 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.720 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T17:18:56.720 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.720 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.721 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.721 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.721 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T17:18:56.721 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.722 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.722 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.722 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.722 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T17:18:56.723 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.723 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.723 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.723 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.724 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T17:18:56.724 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.724 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.724 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.724 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.725 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T17:18:56.725 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.725 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.725 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.725 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.726 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T17:18:56.726 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.726 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.726 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.726 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.727 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T17:18:56.727 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.727 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.727 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.727 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.728 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T17:18:56.728 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.728 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.728 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.728 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.728 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2297, 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 31.037813919, 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.013512326 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T17:18:56.729 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.730 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.730 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.730 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.730 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.730 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T17:18:56.730 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T17:18:56.731 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 5, 2024-05-14T17:18:56.731 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 3, 2024-05-14T17:18:56.731 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-05-14T17:18:56.731 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T17:18:56.732 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 3139, 2024-05-14T17:18:56.732 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 1569, 2024-05-14T17:18:56.732 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 885, 2024-05-14T17:18:56.732 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T17:18:56.732 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:56.733 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:56.734 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:56.735 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:56.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:56.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:56.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:56.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:56.736 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 358638, 2024-05-14T17:18:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 601, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8040, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 168, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 2747565, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 25036, 2024-05-14T17:18:56.739 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T17:18:56.740 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 9500, 2024-05-14T17:18:56.741 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 46260120, 2024-05-14T17:18:56.741 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 28, 2024-05-14T17:18:56.741 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 9500, 2024-05-14T17:18:56.741 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T17:18:56.742 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9500, 2024-05-14T17:18:56.742 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 24.340590560, 2024-05-14T17:18:56.742 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002562167 2024-05-14T17:18:56.742 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.743 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T17:18:56.743 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T17:18:56.743 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9500, 2024-05-14T17:18:56.743 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 10083 2024-05-14T17:18:56.743 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 9500, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 287, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 9213, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-05-14T17:18:56.744 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 282, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 6278, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 199, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 1, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 2319, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T17:18:56.745 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T17:18:56.746 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T17:18:56.747 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T17:18:56.748 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T17:18:56.748 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T17:18:56.748 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T17:18:56.748 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T17:18:56.748 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T17:18:56.749 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T17:18:56.749 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T17:18:56.749 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T17:18:56.749 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T17:18:56.750 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T17:18:56.750 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T17:18:56.750 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T17:18:56.750 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 103, 2024-05-14T17:18:56.751 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T17:18:56.751 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-05-14T17:18:56.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-05-14T17:18:56.751 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-05-14T17:18:56.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-05-14T17:18:56.752 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T17:18:56.752 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 734, 2024-05-14T17:18:56.752 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 10, 2024-05-14T17:18:56.753 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 230 2024-05-14T17:18:56.753 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.753 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T17:18:56.753 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-05-14T17:18:56.753 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:18:56.754 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T17:18:56.754 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T17:18:56.754 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.754 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T17:18:56.754 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 391 2024-05-14T17:18:56.755 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.755 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T17:18:56.755 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T17:18:56.756 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 10, 2024-05-14T17:18:56.756 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T17:18:56.756 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 10, 2024-05-14T17:18:56.756 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 2297, 2024-05-14T17:18:56.756 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 2297, 2024-05-14T17:18:56.757 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T17:18:56.757 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.757 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:56.757 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 170787, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 39353780, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 170787, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:18:56.758 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 170787, 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 39353780, 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.759 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:18:56.760 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 9500, 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 46493557, 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 9500, 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 46493557, 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:56.761 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T17:18:56.762 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 9500, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 9500, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 9500, 2024-05-14T17:18:56.763 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 9500, 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:56.764 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2297, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 233517, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2297, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:18:56.765 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 282, 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 233517, 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.766 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55ee8583a500": { 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 31435, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 45532670, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:56.767 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 31435, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5996, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 45532670, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:56.768 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.769 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:56.769 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T17:18:57.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:56 smithi150 ceph-mon[36204]: pgmap v482: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 55 op/s 2024-05-14T17:18:57.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:56 smithi099 ceph-mon[31851]: pgmap v482: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 55 op/s 2024-05-14T17:18:57.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:56 smithi193 ceph-mon[35962]: pgmap v482: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 55 op/s 2024-05-14T17:18:57.309 INFO:tasks.check_counter:Daemon mds.c mds.exported=12 2024-05-14T17:18:57.310 INFO:tasks.check_counter:Daemon mds.c mds.imported=9 2024-05-14T17:18:57.310 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=90 2024-05-14T17:18:57.310 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=853 2024-05-14T17:18:57.310 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-14T17:18:57.310 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:18:57.310 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:18:57.311 DEBUG:tasks.check_counter:Getting stats from f 2024-05-14T17:18:57.311 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-14T17:18:57.549 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:18:59.052 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T17:18:59.052 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:18:59.052 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 793, 2024-05-14T17:18:59.052 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 793, 2024-05-14T17:18:59.052 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 42188911, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 251426, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.097420130, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.022175871, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.042871138, 2024-05-14T17:18:59.053 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.024841936, 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 792, 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.033231783, 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000041959 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 415, 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 400, 2024-05-14T17:18:59.055 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 41713285, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 159788, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.057219217, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.014836067, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.030968493, 2024-05-14T17:18:59.056 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.009266405, 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 399, 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.014927145, 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000037411 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 271056, 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 77744 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 602, 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 694, 2024-05-14T17:18:59.058 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 64333518, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 440815, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.083935516, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.017094119, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.047678808, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.014466055, 2024-05-14T17:18:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 692, 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.025015463, 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000036149 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.060 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 249696 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T17:18:59.061 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2720, 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.123123150, 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045265 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.062 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.063 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.064 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-05-14T17:18:59.065 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 7361, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 3563, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 3798, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 2271, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-05-14T17:18:59.066 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 20, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-05-14T17:18:59.067 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T17:18:59.068 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 656, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 216471483, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:18:59.069 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:18:59.070 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-05-14T17:18:59.071 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-05-14T17:18:59.072 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T17:18:59.073 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T17:18:59.073 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:18:59.073 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-05-14T17:18:59.073 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-05-14T17:18:59.074 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T17:18:59.074 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T17:18:59.074 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T17:18:59.074 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:18:59.074 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:18:59.074 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T17:18:59.075 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T17:18:59.075 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:18:59.075 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:18:59.075 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:18:59.075 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-05-14T17:18:59.075 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T17:18:59.076 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:18:59.076 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T17:18:59.076 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T17:18:59.076 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.077 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T17:18:59.077 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T17:18:59.077 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-05-14T17:18:59.078 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-05-14T17:18:59.078 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-05-14T17:18:59.078 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 33740, 2024-05-14T17:18:59.078 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 9, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 31, 2024-05-14T17:18:59.079 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T17:18:59.080 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-05-14T17:18:59.080 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 283072929, 2024-05-14T17:18:59.080 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 299059999, 2024-05-14T17:18:59.080 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 299055331, 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 33740 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.081 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T17:18:59.082 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 587, 2024-05-14T17:18:59.082 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 7849, 2024-05-14T17:18:59.082 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 7262, 2024-05-14T17:18:59.082 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 90, 2024-05-14T17:18:59.082 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 170, 2024-05-14T17:18:59.082 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 80, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 7361, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 8593, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 1232, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 63172, 2024-05-14T17:18:59.083 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215392 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T17:18:59.084 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.085 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.086 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.087 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.088 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T17:18:59.089 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.089 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.089 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.089 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.089 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T17:18:59.089 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.090 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.091 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.091 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T17:18:59.091 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.091 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.091 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.091 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.092 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T17:18:59.092 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.093 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.094 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.095 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.096 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.097 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T17:18:59.098 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.099 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T17:18:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.101 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.102 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.103 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T17:18:59.104 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-05-14T17:18:59.105 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-05-14T17:18:59.106 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 0, 2024-05-14T17:18:59.106 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T17:18:59.106 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2399, 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T17:18:59.108 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:18:59.109 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:18:59.110 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:18:59.111 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:18:59.112 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 255694, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 753, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:18:59.113 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8040, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 168, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T17:18:59.114 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 6786712, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 45321, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T17:18:59.115 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 1648, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 26, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 4, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 1648, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1648, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.237979144, 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000751200 2024-05-14T17:18:59.116 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1648, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1649 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 1648, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1648, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-05-14T17:18:59.117 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 921, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 724, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T17:18:59.118 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 2, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-05-14T17:18:59.119 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:18:59.120 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T17:18:59.121 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T17:18:59.122 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 103, 2024-05-14T17:18:59.123 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T17:18:59.123 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 14, 2024-05-14T17:18:59.123 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 10, 2024-05-14T17:18:59.123 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 10, 2024-05-14T17:18:59.125 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-05-14T17:18:59.125 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T17:18:59.125 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-05-14T17:18:59.125 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-05-14T17:18:59.125 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-05-14T17:18:59.125 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T17:18:59.126 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-05-14T17:18:59.127 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1806, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 148062227, 2024-05-14T17:18:59.128 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1806, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1806, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 148062227, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.129 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:59.130 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1648, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 147499403, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1648, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 147499403, 2024-05-14T17:18:59.131 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T17:18:59.132 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1648, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1648, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1648, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1648, 2024-05-14T17:18:59.133 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:59.134 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T17:18:59.135 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55b3ac7b46e0": { 2024-05-14T17:18:59.136 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:18:59.137 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:18:59.138 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T17:18:59.139 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:18:58 smithi193 ceph-mon[35962]: pgmap v483: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 55 op/s 2024-05-14T17:18:59.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:18:58 smithi099 ceph-mon[31851]: pgmap v483: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 55 op/s 2024-05-14T17:18:59.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:18:58 smithi150 ceph-mon[36204]: pgmap v483: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 55 op/s 2024-05-14T17:18:59.836 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-14T17:18:59.836 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-14T17:18:59.837 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-14T17:18:59.837 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-14T17:18:59.837 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-14T17:18:59.837 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:18:59.837 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:18:59.837 DEBUG:tasks.check_counter:Getting stats from i 2024-05-14T17:18:59.837 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-14T17:19:00.089 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:19:01.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:00 smithi099 ceph-mon[31851]: pgmap v484: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.4 MiB/s wr, 58 op/s 2024-05-14T17:19:01.279 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:00 smithi193 ceph-mon[35962]: pgmap v484: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.4 MiB/s wr, 58 op/s 2024-05-14T17:19:01.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:00 smithi150 ceph-mon[36204]: pgmap v484: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.4 MiB/s wr, 58 op/s 2024-05-14T17:19:01.663 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 89722, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 120455, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 22079017, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 149577574, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 188, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 5.608442110, 2024-05-14T17:19:01.664 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 2.334121706, 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 3.101134006, 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.508358139, 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 120454, 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.373695227, 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000036310 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:19:01.665 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 173503, 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.026224209, 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000151 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1932320, 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 206880 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 155623, 2024-05-14T17:19:01.666 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 222903, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 49399972, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 252968137, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 17, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 8.159454987, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 3.594514940, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 4.698940923, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.589152545, 2024-05-14T17:19:01.667 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 222900, 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 9.799671058, 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000043964 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 315767, 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.041665312, 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000131 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.668 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 100432, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 614624 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 79293, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 118485, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 18896917, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 135900225, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-05-14T17:19:01.669 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 6.008867523, 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 2.806867755, 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 2.833864479, 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.599212201, 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 118484, 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6.343822321, 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000053541 2024-05-14T17:19:01.670 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 158071, 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.023801000, 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000150 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 487808, 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 159504 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T17:19:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 59860, 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 52.761362353, 2024-05-14T17:19:01.672 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000881412 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 24402, 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.253151668, 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000133314 2024-05-14T17:19:01.673 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 102834, 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 97796, 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 97796, 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 483.814935583, 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004947185 2024-05-14T17:19:01.674 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 5038, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 14, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 1460, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 7, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 5, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 1586, 2024-05-14T17:19:01.675 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 70, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 30, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 1486, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 1486, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 2086, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 48, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 48, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 20, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 161131, 2024-05-14T17:19:01.676 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 95837, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 5028, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 6, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 11, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 678, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 1360, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T17:19:01.677 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 17, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 1509, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 6, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 260, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 656, 2024-05-14T17:19:01.678 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 216471483, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T17:19:01.679 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 11, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 13257, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 46166, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 26977, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 8220, 2024-05-14T17:19:01.680 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 4153, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 55402 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 221, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 6, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 833, 2024-05-14T17:19:01.681 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 14541, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 14541, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T17:19:01.682 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 12, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 803, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 17, 2024-05-14T17:19:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 12, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 74021, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 67457, 2024-05-14T17:19:01.684 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 67457, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 6564, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 6611, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 163, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 154, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 154, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 9, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 1, 2024-05-14T17:19:01.685 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 10, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 283072929, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 299068881, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 30906, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 94.470908381, 2024-05-14T17:19:01.686 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003056717 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 333, 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 16121, 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 15788, 2024-05-14T17:19:01.687 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 93, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 206, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 113, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 1586, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 35508, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 33922, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 48, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 15702, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 15654, 2024-05-14T17:19:01.688 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 95072, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 231800 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 142132, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 11431, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 102834, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 1572, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 22732, 2024-05-14T17:19:01.689 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 15214, 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 101.993459054, 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.006703921 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 11783, 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 61.958274771, 2024-05-14T17:19:01.690 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005258276 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.691 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 31275, 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 109.934397573, 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003515088 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T17:19:01.692 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.693 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.694 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T17:19:01.694 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.694 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.694 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.694 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.694 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T17:19:01.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.695 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.695 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.695 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T17:19:01.695 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.696 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.696 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.696 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.696 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T17:19:01.696 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 16, 2024-05-14T17:19:01.696 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.006156929, 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000384808 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 119, 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.825644244, 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.006938186 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T17:19:01.697 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 16, 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.161908143, 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.010119258 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.698 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 10, 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.025837339, 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002583733 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 21348, 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 101.778088780, 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004767570 2024-05-14T17:19:01.699 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2735, 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 30.623610470, 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.011196932 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-05-14T17:19:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002464578, 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002464578 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 10, 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.077952716, 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.007795271 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T17:19:01.702 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.016511655, 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003302331 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.703 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 716, 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.040314087, 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001452952 2024-05-14T17:19:01.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 7, 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.019426056, 2024-05-14T17:19:01.705 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002775150 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T17:19:01.706 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 14541, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 75.350889188, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005181960 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.707 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 5, 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 3, 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T17:19:01.708 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 12756, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 6378, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 911, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 360, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 360, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:19:01.709 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:19:01.710 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:19:01.711 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:19:01.712 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 302892, 2024-05-14T17:19:01.713 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 292, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:19:01.714 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8040, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 168, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 2239292, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 24841, 2024-05-14T17:19:01.715 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 56057, 2024-05-14T17:19:01.716 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 308563054, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 37, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 56057, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 56057, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 463.542190300, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.008269122 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T17:19:01.717 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 56057, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 72321 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 56057, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 558, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 55499, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T17:19:01.718 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1268, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 6273, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 544, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 31450, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 546, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 7, 2024-05-14T17:19:01.719 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 15219, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 14, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 12546, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:19:01.720 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T17:19:01.721 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T17:19:01.722 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 103, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-05-14T17:19:01.723 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 3588, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 28, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 838 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-05-14T17:19:01.724 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 475 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 10, 2024-05-14T17:19:01.725 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 10, 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 14541, 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 14541, 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:01.726 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 324634, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 61068593, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 324634, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 324634, 2024-05-14T17:19:01.727 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 61068593, 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:01.728 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 56057, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 310033167, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 56057, 2024-05-14T17:19:01.729 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 310033167, 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T17:19:01.730 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 56057, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 56057, 2024-05-14T17:19:01.731 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 56057, 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 56057, 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:01.732 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 14541, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1470337, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 14541, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:19:01.733 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 544, 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1470337, 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55ee304ba500": { 2024-05-14T17:19:01.734 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 84, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 74021, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 294874577, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 74021, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:19:01.735 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 30906, 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 294874493, 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:01.736 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=221 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=16 2024-05-14T17:19:02.321 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=10 2024-05-14T17:19:02.322 DEBUG:tasks.check_counter:Getting stats from l 2024-05-14T17:19:02.322 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-14T17:19:02.562 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:19:03.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:02 smithi099 ceph-mon[31851]: pgmap v485: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-05-14T17:19:03.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:02 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:19:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:02 smithi193 ceph-mon[35962]: pgmap v485: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-05-14T17:19:03.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:02 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:19:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:02 smithi150 ceph-mon[36204]: pgmap v485: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 71 KiB/s wr, 30 op/s 2024-05-14T17:19:03.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:02 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:19:04.120 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T17:19:04.120 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:19:04.120 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2700, 2024-05-14T17:19:04.120 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2701, 2024-05-14T17:19:04.120 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 25945082, 2024-05-14T17:19:04.120 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 858289, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.241979425, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.072404618, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.072600685, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.077992862, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2699, 2024-05-14T17:19:04.121 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.093650522, 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000034698 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:19:04.123 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:19:04.123 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.123 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:19:04.123 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1072, 2024-05-14T17:19:04.123 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1038, 2024-05-14T17:19:04.123 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 18215210, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 420432, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.126504264, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.035216293, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.070311536, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.026963423, 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:19:04.124 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1037, 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.039020474, 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000037628 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.125 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1867120, 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 199360 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 233, 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 470, 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4255352, 2024-05-14T17:19:04.126 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 649754, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.037471412, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.015177233, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.011228520, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.006661203, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 468, 2024-05-14T17:19:04.127 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.030084212, 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000064282 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.128 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 577568 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.129 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T17:19:04.130 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:19:04.130 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:19:04.130 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6125, 2024-05-14T17:19:04.130 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.156136938, 2024-05-14T17:19:04.130 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000025491 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.131 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.132 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.132 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.132 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.132 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T17:19:04.132 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-05-14T17:19:04.132 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-05-14T17:19:04.133 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T17:19:04.133 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.133 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.133 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.133 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.134 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T17:19:04.134 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-05-14T17:19:04.134 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-05-14T17:19:04.134 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T17:19:04.135 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-05-14T17:19:04.135 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-05-14T17:19:04.135 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-05-14T17:19:04.135 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 3331, 2024-05-14T17:19:04.135 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 78, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 3253, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 3219, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 8, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-05-14T17:19:04.136 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-05-14T17:19:04.137 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-05-14T17:19:04.138 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-05-14T17:19:04.138 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-05-14T17:19:04.138 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T17:19:04.138 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:19:04.138 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T17:19:04.138 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:19:04.139 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:19:04.140 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T17:19:04.140 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:19:04.140 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:19:04.140 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:19:04.140 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:19:04.140 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:19:04.141 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:19:04.141 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-05-14T17:19:04.141 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:19:04.141 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-05-14T17:19:04.142 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-05-14T17:19:04.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.142 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T17:19:04.142 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-05-14T17:19:04.143 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-05-14T17:19:04.143 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-05-14T17:19:04.143 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-05-14T17:19:04.143 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-05-14T17:19:04.144 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T17:19:04.144 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T17:19:04.144 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:19:04.144 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-05-14T17:19:04.145 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-05-14T17:19:04.145 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T17:19:04.145 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T17:19:04.145 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T17:19:04.145 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:19:04.146 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:19:04.146 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T17:19:04.146 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T17:19:04.146 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:19:04.146 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:19:04.147 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:19:04.147 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-05-14T17:19:04.147 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T17:19:04.147 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:19:04.147 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T17:19:04.147 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 31435, 2024-05-14T17:19:04.148 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 38, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 4, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T17:19:04.149 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 49726974, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 49726375, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.150 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 31435 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 666, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 3403, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 2737, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 74, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 88, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 14, 2024-05-14T17:19:04.151 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 3331, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 3765, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 434, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 57960, 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215416 2024-05-14T17:19:04.152 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T17:19:04.153 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.155 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T17:19:04.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.155 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.155 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.156 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.157 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T17:19:04.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.160 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.160 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.160 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.161 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T17:19:04.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.164 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T17:19:04.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.164 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.164 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T17:19:04.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.165 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T17:19:04.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T17:19:04.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.169 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.171 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T17:19:04.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.171 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.172 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T17:19:04.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.172 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.172 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.173 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:19:04.173 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:19:04.173 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T17:19:04.173 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.173 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.174 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.174 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.174 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T17:19:04.174 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T17:19:04.175 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 5, 2024-05-14T17:19:04.175 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 3, 2024-05-14T17:19:04.175 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 4, 2024-05-14T17:19:04.175 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T17:19:04.175 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-05-14T17:19:04.175 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-05-14T17:19:04.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 3885, 2024-05-14T17:19:04.176 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T17:19:04.176 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.176 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T17:19:04.176 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:19:04.177 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T17:19:04.177 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:19:04.177 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:19:04.177 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:19:04.177 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:19:04.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:19:04.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:19:04.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:19:04.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:19:04.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:19:04.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:19:04.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:19:04.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:19:04.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:19:04.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T17:19:04.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:19:04.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T17:19:04.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:19:04.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:19:04.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:19:04.181 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:19:04.181 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:19:04.181 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:19:04.181 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:19:04.181 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T17:19:04.182 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:19:04.182 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:19:04.182 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:19:04.182 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T17:19:04.182 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T17:19:04.183 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T17:19:04.183 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:19:04.183 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:19:04.183 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:19:04.183 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:19:04.184 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 242695, 2024-05-14T17:19:04.184 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 771, 2024-05-14T17:19:04.184 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:19:04.184 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T17:19:04.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T17:19:04.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T17:19:04.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:19:04.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T17:19:04.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:19:04.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T17:19:04.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8040, 2024-05-14T17:19:04.186 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 168, 2024-05-14T17:19:04.186 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:19:04.186 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:19:04.186 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T17:19:04.186 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T17:19:04.186 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 4086651, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 29369, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T17:19:04.187 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3621, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3621, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3621, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.031965831, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000561161 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.188 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3621, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3622 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3621, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 3621, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T17:19:04.189 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 2131, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1488, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T17:19:04.190 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T17:19:04.193 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T17:19:04.193 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-05-14T17:19:04.193 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:19:04.194 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T17:19:04.195 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T17:19:04.196 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 103, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 8, 2024-05-14T17:19:04.197 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 9, 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T17:19:04.198 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T17:19:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:19:04.200 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 4001, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 48033336, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 4001, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 4001, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 48033336, 2024-05-14T17:19:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T17:19:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3621, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 45533842, 2024-05-14T17:19:04.203 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3621, 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 45533842, 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:04.204 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T17:19:04.205 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:04.205 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:19:04.205 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:19:04.205 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:04.205 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:19:04.205 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3621, 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3621, 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3621, 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3621, 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.206 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:04.207 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.208 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5578792986e0": { 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T17:19:04.209 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T17:19:04.210 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T17:19:04.211 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.211 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T17:19:04.211 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T17:19:04.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:03 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:19:04.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:03 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:19:04.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:03 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:19:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:03 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:19:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:03 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:19:04.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:03 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:19:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:03 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:19:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:03 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:19:04.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:03 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:19:04.895 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:19:04.896 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-14T17:19:04.906 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T17:19:04.906 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T17:19:04.906 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:19:04.927 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-14T17:19:04.927 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:19:04.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T17:19:05.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:04 smithi099 ceph-mon[31851]: pgmap v486: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 116 KiB/s wr, 49 op/s 2024-05-14T17:19:05.274 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:05.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:04 smithi193 ceph-mon[35962]: pgmap v486: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 116 KiB/s wr, 49 op/s 2024-05-14T17:19:05.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:04 smithi150 ceph-mon[36204]: pgmap v486: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 116 KiB/s wr, 49 op/s 2024-05-14T17:19:06.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 -- 172.21.15.99:0/865734922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201041a0 msgr2=0x7fdf201081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:06.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 --2- 172.21.15.99:0/865734922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201041a0 0x7fdf201081c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fdf100099c0 tx=0x7fdf10009cd0 comp rx=0 tx=0).stop 2024-05-14T17:19:06.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 -- 172.21.15.99:0/865734922 shutdown_connections 2024-05-14T17:19:06.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 --2- 172.21.15.99:0/865734922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf201088f0 0x7fdf20110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:06.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 --2- 172.21.15.99:0/865734922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf201037f0 0x7fdf20103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:06.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 --2- 172.21.15.99:0/865734922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201041a0 0x7fdf201081c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:06.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 -- 172.21.15.99:0/865734922 >> 172.21.15.99:0/865734922 conn(0x7fdf2007a6a0 msgr2=0x7fdf2007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:06.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 -- 172.21.15.99:0/865734922 shutdown_connections 2024-05-14T17:19:06.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 -- 172.21.15.99:0/865734922 wait complete. 2024-05-14T17:19:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 Processor -- start 2024-05-14T17:19:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.833+0000 7fdf27e91700 1 -- start start 2024-05-14T17:19:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf27e91700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf201037f0 0x7fdf2019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf27e91700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf201041a0 0x7fdf2019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf27e91700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 0x7fdf201999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf27e91700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201136d0 con 0x7fdf201088f0 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf27e91700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20113550 con 0x7fdf201041a0 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf27e91700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20113850 con 0x7fdf201037f0 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 0x7fdf201999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 0x7fdf201999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54424/0 (socket says 172.21.15.99:54424) 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 -- 172.21.15.99:0/1970738525 learned_addr learned my addr 172.21.15.99:0/1970738525 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2542c700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf201041a0 0x7fdf2019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:06.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf25c2d700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf201037f0 0x7fdf2019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 -- 172.21.15.99:0/1970738525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf201037f0 msgr2=0x7fdf2019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf201037f0 0x7fdf2019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 -- 172.21.15.99:0/1970738525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf201041a0 msgr2=0x7fdf2019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf201041a0 0x7fdf2019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 -- 172.21.15.99:0/1970738525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2019a110 con 0x7fdf201088f0 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf2642e700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 0x7fdf201999b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800ada0 tx=0x7fdf1800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.834+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18017020 con 0x7fdf201088f0 2024-05-14T17:19:06.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.835+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2019a400 con 0x7fdf201088f0 2024-05-14T17:19:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.835+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1800cd80 con 0x7fdf201088f0 2024-05-14T17:19:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.835+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18005260 con 0x7fdf201088f0 2024-05-14T17:19:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.835+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a66c0 con 0x7fdf201088f0 2024-05-14T17:19:06.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.835+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2006a230 con 0x7fdf201088f0 2024-05-14T17:19:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.837+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdf1800a090 con 0x7fdf201088f0 2024-05-14T17:19:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.838+0000 7fdf16ffd700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf0c0b29f0 0x7fdf0c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.838+0000 7fdf25c2d700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf0c0b29f0 0x7fdf0c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.838+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fdf180e22a0 con 0x7fdf201088f0 2024-05-14T17:19:06.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.838+0000 7fdf25c2d700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf0c0b29f0 0x7fdf0c0b4eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf2019aaf0 tx=0x7fdf1c00a380 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:06.841+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf18021020 con 0x7fdf201088f0 2024-05-14T17:19:07.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.057+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf2002e560 con 0x7fdf201088f0 2024-05-14T17:19:07.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.057+0000 7fdf16ffd700 1 -- 172.21.15.99:0/1970738525 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+942 (secure 0 0 0) 0x7fdf180a9cb0 con 0x7fdf201088f0 2024-05-14T17:19:07.058 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/2498052825 2024-05-15T17:03:37.568496+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/2498052825 2024-05-15T17:03:37.568496+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1935223933 2024-05-15T17:03:37.568496+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/575236794 2024-05-15T16:53:02.369520+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/488380564 2024-05-15T16:53:02.369520+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/1504956360 2024-05-15T16:52:36.088176+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/3420859495 2024-05-15T16:52:36.088176+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2756368842 2024-05-15T16:52:36.088176+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/1504956360 2024-05-15T16:52:36.088176+0000 2024-05-14T17:19:07.059 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/781830097 2024-05-15T17:03:37.568496+0000 2024-05-14T17:19:07.060 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/701736301 2024-05-15T17:03:37.568496+0000 2024-05-14T17:19:07.060 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/1816772983 2024-05-15T16:53:02.369520+0000 2024-05-14T17:19:07.060 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/3450933746 2024-05-15T16:52:36.088176+0000 2024-05-14T17:19:07.060 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2842786850 2024-05-15T17:03:37.568496+0000 2024-05-14T17:19:07.060 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1463965565 2024-05-15T16:53:02.369520+0000 2024-05-14T17:19:07.060 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/1816772983 2024-05-15T16:53:02.369520+0000 2024-05-14T17:19:07.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf0c0b29f0 msgr2=0x7fdf0c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:07.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf0c0b29f0 0x7fdf0c0b4eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf2019aaf0 tx=0x7fdf1c00a380 comp rx=0 tx=0).stop 2024-05-14T17:19:07.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 msgr2=0x7fdf201999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:07.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 0x7fdf201999b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800ada0 tx=0x7fdf1800c330 comp rx=0 tx=0).stop 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 shutdown_connections 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fdf0c0b29f0 0x7fdf0c0b4eb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf201088f0 0x7fdf201999b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf201037f0 0x7fdf2019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 --2- 172.21.15.99:0/1970738525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf201041a0 0x7fdf2019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 >> 172.21.15.99:0/1970738525 conn(0x7fdf2007a6a0 msgr2=0x7fdf20101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 shutdown_connections 2024-05-14T17:19:07.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:07.060+0000 7fdf27e91700 1 -- 172.21.15.99:0/1970738525 wait complete. 2024-05-14T17:19:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:listed 16 entries 2024-05-14T17:19:07.092 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:06 smithi099 ceph-mon[31851]: pgmap v487: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-05-14T17:19:07.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:06 smithi193 ceph-mon[35962]: pgmap v487: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-05-14T17:19:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:06 smithi150 ceph-mon[36204]: pgmap v487: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-05-14T17:19:07.780 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-14T17:19:07.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:19:07.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-14T17:19:07.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:19:07.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:19:07.878 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T17:19:07.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:19:07.930 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-05-14T17:19:07.970 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T17:19:07.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:19:07.971 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T17:19:08.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:19:08.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-05-14T17:19:08.150 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-14T17:19:08.160 INFO:tasks.cephadm:Teardown begin 2024-05-14T17:19:08.160 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T17:19:08.161 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:19:08.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:08 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1970738525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:19:08.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:08 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1970738525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:19:08.407 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:08.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:08 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1970738525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:19:09.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:09 smithi099 ceph-mon[31851]: pgmap v488: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-05-14T17:19:09.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:09 smithi193 ceph-mon[35962]: pgmap v488: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-05-14T17:19:09.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:09 smithi150 ceph-mon[36204]: pgmap v488: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-05-14T17:19:09.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2966267802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 msgr2=0x7f9d0406c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:09.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2966267802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d0406c150 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009990 tx=0x7f9cf4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2966267802 shutdown_connections 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2966267802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d0406c690 0x7f9d041103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2966267802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d041095b0 0x7f9d04109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2966267802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d0406c150 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2966267802 >> 172.21.15.99:0/2966267802 conn(0x7f9d040793f0 msgr2=0x7f9d04079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2966267802 shutdown_connections 2024-05-14T17:19:09.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.954+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2966267802 wait complete. 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 Processor -- start 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 -- start start 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d04102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d0406c690 0x7f9d04102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d041095b0 0x7f9d04102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041137c0 con 0x7f9d0406bcd0 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113640 con 0x7f9d041095b0 2024-05-14T17:19:09.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0bec9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04113940 con 0x7f9d0406c690 2024-05-14T17:19:09.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d04102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:09.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d04102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54446/0 (socket says 172.21.15.99:54446) 2024-05-14T17:19:09.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 -- 172.21.15.99:0/2008551083 learned_addr learned my addr 172.21.15.99:0/2008551083 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:09.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d0a466700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d041095b0 0x7f9d04102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:09.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09464700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d0406c690 0x7f9d04102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:09.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 -- 172.21.15.99:0/2008551083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d0406c690 msgr2=0x7f9d04102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d0406c690 0x7f9d04102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 -- 172.21.15.99:0/2008551083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d041095b0 msgr2=0x7f9d04102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d041095b0 0x7f9d04102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 -- 172.21.15.99:0/2008551083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d041a8300 con 0x7f9d0406bcd0 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09464700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d0406c690 0x7f9d04102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.955+0000 7f9d09c65700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d04102510 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000ba50 tx=0x7f9d0000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.956+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00019070 con 0x7f9d0406bcd0 2024-05-14T17:19:09.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.956+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d041a85f0 con 0x7f9d0406bcd0 2024-05-14T17:19:09.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.956+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d00014440 con 0x7f9d0406bcd0 2024-05-14T17:19:09.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.956+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000ce60 con 0x7f9d0406bcd0 2024-05-14T17:19:09.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.956+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041a8b30 con 0x7f9d0406bcd0 2024-05-14T17:19:09.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.957+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d0410ba90 con 0x7f9d0406bcd0 2024-05-14T17:19:09.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.958+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9d0001c070 con 0x7f9d0406bcd0 2024-05-14T17:19:09.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.959+0000 7f9cfaffd700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9cf00b29f0 0x7f9cf00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:09.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.959+0000 7f9d09464700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9cf00b29f0 0x7f9cf00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:09.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.959+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f9d000e1dd0 con 0x7f9d0406bcd0 2024-05-14T17:19:09.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.959+0000 7f9d09464700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9cf00b29f0 0x7f9cf00b4eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b5c0 tx=0x7f9cf4005fb0 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:09.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:09.963+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d000a98c0 con 0x7f9d0406bcd0 2024-05-14T17:19:10.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.215+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d0406a230 con 0x7f9d0406bcd0 2024-05-14T17:19:10.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.216+0000 7f9cfaffd700 1 -- 172.21.15.99:0/2008551083 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11880 (secure 0 0 0) 0x7f9d000a91c0 con 0x7f9d0406bcd0 2024-05-14T17:19:10.217 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:19:10.217 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:18:39.600304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:19:10.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9cf00b29f0 msgr2=0x7f9cf00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:10.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9cf00b29f0 0x7f9cf00b4eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b5c0 tx=0x7f9cf4005fb0 comp rx=0 tx=0).stop 2024-05-14T17:19:10.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 msgr2=0x7f9d04102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:10.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d04102510 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000ba50 tx=0x7f9d0000bd60 comp rx=0 tx=0).stop 2024-05-14T17:19:10.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 shutdown_connections 2024-05-14T17:19:10.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9cf00b29f0 0x7f9cf00b4eb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:10.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d041095b0 0x7f9d04102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:10.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406bcd0 0x7f9d04102510 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:10.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 --2- 172.21.15.99:0/2008551083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d0406c690 0x7f9d04102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:10.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 >> 172.21.15.99:0/2008551083 conn(0x7f9d040793f0 msgr2=0x7f9d04101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:10.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 shutdown_connections 2024-05-14T17:19:10.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:10.218+0000 7f9d0bec9700 1 -- 172.21.15.99:0/2008551083 wait complete. 2024-05-14T17:19:10.221 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 47 2024-05-14T17:19:10.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:19:11.122 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:11.262 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:11 smithi099 ceph-mon[31851]: pgmap v489: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 27 op/s 2024-05-14T17:19:11.262 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:11 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2008551083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:19:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:11 smithi193 ceph-mon[35962]: pgmap v489: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 27 op/s 2024-05-14T17:19:11.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:11 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2008551083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:19:11.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:11 smithi150 ceph-mon[36204]: pgmap v489: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 27 op/s 2024-05-14T17:19:11.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:11 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2008551083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:19:12.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:12 smithi099 ceph-mon[31851]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:19:12.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:12 smithi193 ceph-mon[35962]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:19:12.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:12 smithi150 ceph-mon[36204]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:19:12.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 -- 172.21.15.99:0/281664859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 msgr2=0x7f53d8103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:12.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 --2- 172.21.15.99:0/281664859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d8103c00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0099c0 tx=0x7f53cc009cd0 comp rx=0 tx=0).stop 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 -- 172.21.15.99:0/281664859 shutdown_connections 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 --2- 172.21.15.99:0/281664859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d8108910 0x7f53d8110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 --2- 172.21.15.99:0/281664859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d8103c00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 --2- 172.21.15.99:0/281664859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d81041d0 0x7f53d81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.763+0000 7f53dcd02700 1 -- 172.21.15.99:0/281664859 >> 172.21.15.99:0/281664859 conn(0x7f53d807a6b0 msgr2=0x7f53d807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.764+0000 7f53dcd02700 1 -- 172.21.15.99:0/281664859 shutdown_connections 2024-05-14T17:19:12.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.764+0000 7f53dcd02700 1 -- 172.21.15.99:0/281664859 wait complete. 2024-05-14T17:19:12.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.764+0000 7f53dcd02700 1 Processor -- start 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 -- start start 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d81041d0 0x7f53d819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d8108910 0x7f53d8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8113750 con 0x7f53d8103820 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81135d0 con 0x7f53d8108910 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53dcd02700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53d81138d0 con 0x7f53d81041d0 2024-05-14T17:19:12.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54464/0 (socket says 172.21.15.99:54464) 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 -- 172.21.15.99:0/52619720 learned_addr learned my addr 172.21.15.99:0/52619720 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d5d9b700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d81041d0 0x7f53d819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d6d9d700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d8108910 0x7f53d8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 -- 172.21.15.99:0/52619720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d81041d0 msgr2=0x7f53d819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d81041d0 0x7f53d819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 -- 172.21.15.99:0/52619720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d8108910 msgr2=0x7f53d8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:12.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d8108910 0x7f53d8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 -- 172.21.15.99:0/52619720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d819a170 con 0x7f53d8103820 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.765+0000 7f53d659c700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d819f850 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00b580 tx=0x7f53cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.766+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc027070 con 0x7f53d8103820 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.766+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc0040a0 con 0x7f53d8103820 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.766+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d819a400 con 0x7f53d8103820 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.766+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00f460 con 0x7f53d8103820 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.766+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d81a6720 con 0x7f53d8103820 2024-05-14T17:19:12.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.767+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d806a230 con 0x7f53d8103820 2024-05-14T17:19:12.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.768+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f53cc02a070 con 0x7f53d8103820 2024-05-14T17:19:12.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.769+0000 7f53c77fe700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f53c00b2a70 0x7f53c00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:12.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.769+0000 7f53d5d9b700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f53c00b2a70 0x7f53c00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:12.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.769+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8565+0+0 (secure 0 0 0) 0x7f53cc0e3b40 con 0x7f53d8103820 2024-05-14T17:19:12.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.769+0000 7f53d5d9b700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f53c00b2a70 0x7f53c00b4f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53d8103190 tx=0x7f53c8009190 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:12.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:12.772+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53cc031070 con 0x7f53d8103820 2024-05-14T17:19:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.018+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53d819af40 con 0x7f53d8103820 2024-05-14T17:19:13.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.021+0000 7f53c77fe700 1 -- 172.21.15.99:0/52619720 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11880 (secure 0 0 0) 0x7f53cc013070 con 0x7f53d8103820 2024-05-14T17:19:13.021 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:19:13.021 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3118783767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3118783767},{"type":"v1","addr":"172.21.15.99:6835","nonce":3118783767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2633931884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2633931884},{"type":"v1","addr":"172.21.15.99:6837","nonce":2633931884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/913097680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":913097680},{"type":"v1","addr":"172.21.15.99:6839","nonce":913097680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/862670841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":862670841},{"type":"v1","addr":"172.21.15.99:6841","nonce":862670841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/308217426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":308217426},{"type":"v1","addr":"172.21.15.150:6833","nonce":308217426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/955154875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":955154875},{"type":"v1","addr":"172.21.15.150:6835","nonce":955154875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T17:00:11.851265+0000","modified":"2024-05-14T17:18:39.600304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24463,"mds_1":24448,"mds_2":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/1891176204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1891176204},{"type":"v1","addr":"172.21.15.150:6837","nonce":1891176204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3825469703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3825469703},{"type":"v1","addr":"172.21.15.193:6833","nonce":3825469703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.150:6839/3105942045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":3105942045},{"type":"v1","addr":"172.21.15.150:6839","nonce":3105942045}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.193:6837/4135945134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4135945134},{"type":"v1","addr":"172.21.15.193:6837","nonce":4135945134}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/4034311425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":4034311425},{"type":"v1","addr":"172.21.15.193:6839","nonce":4034311425}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25687":{"gid":25687,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4018228315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4018228315},{"type":"v1","addr":"172.21.15.193:6835","nonce":4018228315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24463,"qdb_cluster":[24443,24448,24463]},"id":1}]} 2024-05-14T17:19:13.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f53c00b2a70 msgr2=0x7f53c00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:13.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f53c00b2a70 0x7f53c00b4f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53d8103190 tx=0x7f53c8009190 comp rx=0 tx=0).stop 2024-05-14T17:19:13.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 msgr2=0x7f53d819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:13.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d819f850 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00b580 tx=0x7f53cc0047b0 comp rx=0 tx=0).stop 2024-05-14T17:19:13.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 shutdown_connections 2024-05-14T17:19:13.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f53c00b2a70 0x7f53c00b4f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:13.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d8108910 0x7f53d8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:13.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8103820 0x7f53d819f850 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:13.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 --2- 172.21.15.99:0/52619720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d81041d0 0x7f53d819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:13.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.023+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 >> 172.21.15.99:0/52619720 conn(0x7f53d807a6b0 msgr2=0x7f53d81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:13.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.024+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 shutdown_connections 2024-05-14T17:19:13.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:13.024+0000 7f53dcd02700 1 -- 172.21.15.99:0/52619720 wait complete. 2024-05-14T17:19:13.025 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 47 2024-05-14T17:19:13.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:13 smithi193 ceph-mon[35962]: pgmap v491: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-05-14T17:19:13.339 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:13 smithi099 ceph-mon[31851]: pgmap v491: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-05-14T17:19:13.354 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:13 smithi150 ceph-mon[36204]: pgmap v491: 97 pgs: 97 active+clean; 394 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-05-14T17:19:13.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:19:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:14.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:14 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/52619720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:19:14.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:14 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/52619720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:19:14.366 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:14 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/52619720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:19:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:15 smithi099 ceph-mon[31851]: pgmap v492: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-05-14T17:19:15.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:15 smithi099 ceph-mon[31851]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:19:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:15 smithi193 ceph-mon[35962]: pgmap v492: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-05-14T17:19:15.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:15 smithi193 ceph-mon[35962]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:19:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:15 smithi150 ceph-mon[36204]: pgmap v492: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-05-14T17:19:15.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:15 smithi150 ceph-mon[36204]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:19:15.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 -- 172.21.15.99:0/1278639563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c108510 msgr2=0x7f9f4c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 --2- 172.21.15.99:0/1278639563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c108510 0x7f9f4c1088f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b0a0 tx=0x7f9f4400b3b0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 -- 172.21.15.99:0/1278639563 shutdown_connections 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 --2- 172.21.15.99:0/1278639563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f4c102df0 0x7f9f4c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 --2- 172.21.15.99:0/1278639563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c108510 0x7f9f4c1088f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 --2- 172.21.15.99:0/1278639563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f4c102430 0x7f9f4c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 -- 172.21.15.99:0/1278639563 >> 172.21.15.99:0/1278639563 conn(0x7f9f4c0793b0 msgr2=0x7f9f4c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 -- 172.21.15.99:0/1278639563 shutdown_connections 2024-05-14T17:19:15.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.535+0000 7f9f535b2700 1 -- 172.21.15.99:0/1278639563 wait complete. 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 Processor -- start 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 -- start start 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 0x7f9f4c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f4c102df0 0x7f9f4c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f4c108510 0x7f9f4c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1125c0 con 0x7f9f4c102430 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c112440 con 0x7f9f4c108510 2024-05-14T17:19:15.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f535b2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c112740 con 0x7f9f4c102df0 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f5134e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 0x7f9f4c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f5134e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 0x7f9f4c19f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53394/0 (socket says 172.21.15.99:53394) 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f5134e700 1 -- 172.21.15.99:0/2828129362 learned_addr learned my addr 172.21.15.99:0/2828129362 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.536+0000 7f9f51b4f700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f4c108510 0x7f9f4c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f50b4d700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f4c102df0 0x7f9f4c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f5134e700 1 -- 172.21.15.99:0/2828129362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f4c102df0 msgr2=0x7f9f4c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f5134e700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f4c102df0 0x7f9f4c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f5134e700 1 -- 172.21.15.99:0/2828129362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f4c108510 msgr2=0x7f9f4c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:15.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f5134e700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f4c108510 0x7f9f4c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f5134e700 1 -- 172.21.15.99:0/2828129362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c19a1d0 con 0x7f9f4c102430 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f5134e700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 0x7f9f4c19f8b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b070 tx=0x7f9f44004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44021070 con 0x7f9f4c102430 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f44007e30 con 0x7f9f4c102430 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c19a460 con 0x7f9f4c102430 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440046c0 con 0x7f9f4c102430 2024-05-14T17:19:15.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.537+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c1a6780 con 0x7f9f4c102430 2024-05-14T17:19:15.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.538+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c06a230 con 0x7f9f4c102430 2024-05-14T17:19:15.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.539+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9f44007980 con 0x7f9f4c102430 2024-05-14T17:19:15.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.540+0000 7f9f4a7fc700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9f340b29f0 0x7f9f340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:15.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.540+0000 7f9f50b4d700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9f340b29f0 0x7f9f340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:15.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.541+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7f9f4400f020 con 0x7f9f4c102430 2024-05-14T17:19:15.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.541+0000 7f9f50b4d700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9f340b29f0 0x7f9f340b4eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c079940 tx=0x7f9f40009260 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:15.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.544+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f4402a020 con 0x7f9f4c102430 2024-05-14T17:19:15.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.751+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f4c1a6950 con 0x7f9f4c102430 2024-05-14T17:19:15.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.752+0000 7f9f4a7fc700 1 -- 172.21.15.99:0/2828129362 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+23165 (secure 0 0 0) 0x7f9f440ab8c0 con 0x7f9f4c102430 2024-05-14T17:19:15.752 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-14T17:19:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":105,"fsid":"2a006d80-1212-11ef-bc98-c7b262605968","created":"2024-05-14T16:52:09.109385+0000","modified":"2024-05-14T17:19:14.972269+0000","last_up_change":"2024-05-14T16:59:03.602779+0000","last_in_change":"2024-05-14T16:58:43.904093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:55:26.450956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T17:00:05.990282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T17:00:08.701546+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0db35f1-6897-4148-8abf-3d8f5945234d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6803","nonce":3363601004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6805","nonce":3363601004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6809","nonce":3363601004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3363601004},{"type":"v1","addr":"172.21.15.99:6807","nonce":3363601004}]},"public_addr":"172.21.15.99:6803/3363601004","cluster_addr":"172.21.15.99:6805/3363601004","heartbeat_back_addr":"172.21.15.99:6809/3363601004","heartbeat_front_addr":"172.21.15.99:6807/3363601004","state":["exists","up"]},{"osd":1,"uuid":"9bc6113d-d89d-4515-a997-f001e08ff626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6801","nonce":3718481884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6803","nonce":3718481884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6807","nonce":3718481884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3718481884},{"type":"v1","addr":"172.21.15.150:6805","nonce":3718481884}]},"public_addr":"172.21.15.150:6801/3718481884","cluster_addr":"172.21.15.150:6803/3718481884","heartbeat_back_addr":"172.21.15.150:6807/3718481884","heartbeat_front_addr":"172.21.15.150:6805/3718481884","state":["exists","up"]},{"osd":2,"uuid":"4c2f82f4-d4f0-4b01-9fb7-6fa62787b0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6801","nonce":2485896294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6803","nonce":2485896294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6807","nonce":2485896294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2485896294},{"type":"v1","addr":"172.21.15.193:6805","nonce":2485896294}]},"public_addr":"172.21.15.193:6801/2485896294","cluster_addr":"172.21.15.193:6803/2485896294","heartbeat_back_addr":"172.21.15.193:6807/2485896294","heartbeat_front_addr":"172.21.15.193:6805/2485896294","state":["exists","up"]},{"osd":3,"uuid":"ad8e6b76-114d-4622-8e95-ea570fb85003","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6811","nonce":429217969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6813","nonce":429217969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6817","nonce":429217969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":429217969},{"type":"v1","addr":"172.21.15.99:6815","nonce":429217969}]},"public_addr":"172.21.15.99:6811/429217969","cluster_addr":"172.21.15.99:6813/429217969","heartbeat_back_addr":"172.21.15.99:6817/429217969","heartbeat_front_addr":"172.21.15.99:6815/429217969","state":["exists","up"]},{"osd":4,"uuid":"cf410b8e-faab-433d-8af0-db09e90b91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6809","nonce":1354191038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6811","nonce":1354191038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6815","nonce":1354191038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1354191038},{"type":"v1","addr":"172.21.15.150:6813","nonce":1354191038}]},"public_addr":"172.21.15.150:6809/1354191038","cluster_addr":"172.21.15.150:6811/1354191038","heartbeat_back_addr":"172.21.15.150:6815/1354191038","heartbeat_front_addr":"172.21.15.150:6813/1354191038","state":["exists","up"]},{"osd":5,"uuid":"1e5a17c9-2b6a-4cdd-bc67-9d18ec898efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6809","nonce":2563526327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6811","nonce":2563526327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6815","nonce":2563526327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2563526327},{"type":"v1","addr":"172.21.15.193:6813","nonce":2563526327}]},"public_addr":"172.21.15.193:6809/2563526327","cluster_addr":"172.21.15.193:6811/2563526327","heartbeat_back_addr":"172.21.15.193:6815/2563526327","heartbeat_front_addr":"172.21.15.193:6813/2563526327","state":["exists","up"]},{"osd":6,"uuid":"4a01d0f1-49fc-4260-92d1-e75e3ec36b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6819","nonce":3623238310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6821","nonce":3623238310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6825","nonce":3623238310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3623238310},{"type":"v1","addr":"172.21.15.99:6823","nonce":3623238310}]},"public_addr":"172.21.15.99:6819/3623238310","cluster_addr":"172.21.15.99:6821/3623238310","heartbeat_back_addr":"172.21.15.99:6825/3623238310","heartbeat_front_addr":"172.21.15.99:6823/3623238310","state":["exists","up"]},{"osd":7,"uuid":"77c663e2-65e8-46c2-9363-bf3fecf22dd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6817","nonce":1186953586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6819","nonce":1186953586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6823","nonce":1186953586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1186953586},{"type":"v1","addr":"172.21.15.150:6821","nonce":1186953586}]},"public_addr":"172.21.15.150:6817/1186953586","cluster_addr":"172.21.15.150:6819/1186953586","heartbeat_back_addr":"172.21.15.150:6823/1186953586","heartbeat_front_addr":"172.21.15.150:6821/1186953586","state":["exists","up"]},{"osd":8,"uuid":"0e6281da-4f42-4ba0-8033-cac6194b5a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6817","nonce":2890842755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6819","nonce":2890842755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6823","nonce":2890842755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2890842755},{"type":"v1","addr":"172.21.15.193:6821","nonce":2890842755}]},"public_addr":"172.21.15.193:6817/2890842755","cluster_addr":"172.21.15.193:6819/2890842755","heartbeat_back_addr":"172.21.15.193:6823/2890842755","heartbeat_front_addr":"172.21.15.193:6821/2890842755","state":["exists","up"]},{"osd":9,"uuid":"302e14e8-2084-4de2-947a-3df7b3f8e45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6827","nonce":3111395354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6829","nonce":3111395354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6833","nonce":3111395354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3111395354},{"type":"v1","addr":"172.21.15.99:6831","nonce":3111395354}]},"public_addr":"172.21.15.99:6827/3111395354","cluster_addr":"172.21.15.99:6829/3111395354","heartbeat_back_addr":"172.21.15.99:6833/3111395354","heartbeat_front_addr":"172.21.15.99:6831/3111395354","state":["exists","up"]},{"osd":10,"uuid":"64fe2c42-8ae1-4465-a821-ff6118d2a262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6825","nonce":2780871308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6827","nonce":2780871308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6831","nonce":2780871308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2780871308},{"type":"v1","addr":"172.21.15.150:6829","nonce":2780871308}]},"public_addr":"172.21.15.150:6825/2780871308","cluster_addr":"172.21.15.150:6827/2780871308","heartbeat_back_addr":"172.21.15.150:6831/2780871308","heartbeat_front_addr":"172.21.15.150:6829/2780871308","state":["exists","up"]},{"osd":11,"uuid":"cc26027a-0163-4ecd-88da-2029accfe161","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6825","nonce":3625177067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6827","nonce":3625177067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6831","nonce":3625177067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3625177067},{"type":"v1","addr":"172.21.15.193:6829","nonce":3625177067}]},"public_addr":"172.21.15.193:6825/3625177067","cluster_addr":"172.21.15.193:6827/3625177067","heartbeat_back_addr":"172.21.15.193:6831/3625177067","heartbeat_front_addr":"172.21.15.193:6829/3625177067","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:35.725523+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:54:59.366595+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:23.237993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:55:46.959029+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:10.624238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:34.709098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:56:58.367379+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:22.919226+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:57:46.902299+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:11.748036+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:58:37.067298+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:59:01.851591+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":7},{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/2498052825":"2024-05-15T17:03:37.568496+0000","172.21.15.99:6800/2498052825":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/1935223933":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/575236794":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/488380564":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/3420859495":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2756368842":"2024-05-15T16:52:36.088176+0000","172.21.15.99:6801/1504956360":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/781830097":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/701736301":"2024-05-15T17:03:37.568496+0000","172.21.15.99:6801/1816772983":"2024-05-15T16:53:02.369520+0000","172.21.15.99:0/3450933746":"2024-05-15T16:52:36.088176+0000","172.21.15.99:0/2842786850":"2024-05-15T17:03:37.568496+0000","172.21.15.99:0/1463965565":"2024-05-15T16:53:02.369520+0000","172.21.15.99:6800/1816772983":"2024-05-15T16:53:02.369520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:19:15.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9f340b29f0 msgr2=0x7f9f340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:15.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9f340b29f0 0x7f9f340b4eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c079940 tx=0x7f9f40009260 comp rx=0 tx=0).stop 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 msgr2=0x7f9f4c19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 0x7f9f4c19f8b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b070 tx=0x7f9f44004d00 comp rx=0 tx=0).stop 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 shutdown_connections 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f9f340b29f0 0x7f9f340b4eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f4c108510 0x7f9f4c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f4c102430 0x7f9f4c19f8b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 --2- 172.21.15.99:0/2828129362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f4c102df0 0x7f9f4c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 >> 172.21.15.99:0/2828129362 conn(0x7f9f4c0793b0 msgr2=0x7f9f4c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:15.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 shutdown_connections 2024-05-14T17:19:15.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:15.755+0000 7f9f535b2700 1 -- 172.21.15.99:0/2828129362 wait complete. 2024-05-14T17:19:16.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:16 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2828129362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:19:16.412 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:16 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2828129362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:19:16.438 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-14T17:19:16.452 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:16 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2828129362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:19:16.677 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:17.165 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:17 smithi099 ceph-mon[31851]: pgmap v494: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-05-14T17:19:17.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:17 smithi193 ceph-mon[35962]: pgmap v494: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-05-14T17:19:17.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:17 smithi150 ceph-mon[36204]: pgmap v494: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-05-14T17:19:18.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 -- 172.21.15.99:0/4155845937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 msgr2=0x7fe804104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 --2- 172.21.15.99:0/4155845937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe804104630 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe8000099c0 tx=0x7fe800009cd0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 -- 172.21.15.99:0/4155845937 shutdown_connections 2024-05-14T17:19:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 --2- 172.21.15.99:0/4155845937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 0x7fe804112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 --2- 172.21.15.99:0/4155845937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe804104630 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 --2- 172.21.15.99:0/4155845937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe804101d70 0x7fe8041021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 -- 172.21.15.99:0/4155845937 >> 172.21.15.99:0/4155845937 conn(0x7fe8040776f0 msgr2=0x7fe804077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 -- 172.21.15.99:0/4155845937 shutdown_connections 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.581+0000 7fe80a79e700 1 -- 172.21.15.99:0/4155845937 wait complete. 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 Processor -- start 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 -- start start 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe804101d70 0x7fe80419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 0x7fe80419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe8041a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe804115a60 con 0x7fe804104250 2024-05-14T17:19:18.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041158e0 con 0x7fe804101d70 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80a79e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe804115be0 con 0x7fe804102730 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe8041a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe8041a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53408/0 (socket says 172.21.15.99:53408) 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 -- 172.21.15.99:0/1950229156 learned_addr learned my addr 172.21.15.99:0/1950229156 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe80979c700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe804101d70 0x7fe80419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe808f9b700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 0x7fe80419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:18.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 -- 172.21.15.99:0/1950229156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 msgr2=0x7fe80419c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 0x7fe80419c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 -- 172.21.15.99:0/1950229156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe804101d70 msgr2=0x7fe80419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe804101d70 0x7fe80419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe809f9d700 1 -- 172.21.15.99:0/1950229156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8041a0ea0 con 0x7fe804104250 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.582+0000 7fe808f9b700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 0x7fe80419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.583+0000 7fe809f9d700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe8041a0760 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc007d70 tx=0x7fe7fc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.583+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc016020 con 0x7fe804104250 2024-05-14T17:19:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.583+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8041a1190 con 0x7fe804104250 2024-05-14T17:19:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.583+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7fc00ad40 con 0x7fe804104250 2024-05-14T17:19:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.583+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0115e0 con 0x7fe804104250 2024-05-14T17:19:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.583+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041a8a10 con 0x7fe804104250 2024-05-14T17:19:18.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.584+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80406a230 con 0x7fe804104250 2024-05-14T17:19:18.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.585+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe7fc019020 con 0x7fe804104250 2024-05-14T17:19:18.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.586+0000 7fe7fa7fc700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe7f00b29f0 0x7fe7f00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:18.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.586+0000 7fe80979c700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe7f00b29f0 0x7fe7f00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:18.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.586+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7fe7fc0e2a00 con 0x7fe804104250 2024-05-14T17:19:18.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.587+0000 7fe80979c700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe7f00b29f0 0x7fe7f00b4eb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe800009990 tx=0x7fe80000b870 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.590+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7fc020020 con 0x7fe804104250 2024-05-14T17:19:18.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.840+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe80419d1f0 con 0x7fe804104250 2024-05-14T17:19:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.842+0000 7fe7fa7fc700 1 -- 172.21.15.99:0/1950229156 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7fe7fc0aa4f0 con 0x7fe804104250 2024-05-14T17:19:18.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe7f00b29f0 msgr2=0x7fe7f00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:18.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe7f00b29f0 0x7fe7f00b4eb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe800009990 tx=0x7fe80000b870 comp rx=0 tx=0).stop 2024-05-14T17:19:18.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 msgr2=0x7fe8041a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:18.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe8041a0760 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc007d70 tx=0x7fe7fc00a340 comp rx=0 tx=0).stop 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 shutdown_connections 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fe7f00b29f0 0x7fe7f00b4eb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe804104250 0x7fe8041a0760 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe804101d70 0x7fe80419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 --2- 172.21.15.99:0/1950229156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804102730 0x7fe80419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 >> 172.21.15.99:0/1950229156 conn(0x7fe8040776f0 msgr2=0x7fe8040ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.845+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 shutdown_connections 2024-05-14T17:19:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:18.846+0000 7fe80a79e700 1 -- 172.21.15.99:0/1950229156 wait complete. 2024-05-14T17:19:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T17:19:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: pgmap v495: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-14T17:19:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1950229156' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:19:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:19:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:19:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/1950229156' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:19:19.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:19:19.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:19 smithi150 ceph-mon[36204]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T17:19:19.204 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:19:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b[66605]: ignoring --setuser ceph since I am not root 2024-05-14T17:19:19.204 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:19:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b[66605]: ignoring --setgroup ceph since I am not root 2024-05-14T17:19:19.204 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:19:18 smithi150 ceph-mds[66609]: -- 172.21.15.150:0/4097525043 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56174a961a40 con 0x56174a935000 2024-05-14T17:19:19.204 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:19:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b[66605]: starting mds.b at 2024-05-14T17:19:19.205 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:19:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h[66289]: ignoring --setuser ceph since I am not root 2024-05-14T17:19:19.205 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:19:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h[66289]: ignoring --setgroup ceph since I am not root 2024-05-14T17:19:19.205 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:19:18 smithi150 ceph-mds[66293]: -- 172.21.15.150:0/3145403087 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591335bba40 con 0x55913358e800 2024-05-14T17:19:19.205 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:19:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h[66289]: starting mds.h at 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a[31847]: 2024-05-14T17:19:18.841+0000 7fd0747be700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: pgmap v495: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1950229156' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/1950229156' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:19:19.207 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:19:19.208 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:19 smithi099 ceph-mon[31851]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T17:19:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: pgmap v495: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-14T17:19:19.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1950229156' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/1950229156' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:19 smithi193 ceph-mon[35962]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c[67374]: ignoring --setuser ceph since I am not root 2024-05-14T17:19:19.293 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c[67374]: ignoring --setgroup ceph since I am not root 2024-05-14T17:19:19.294 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c[67374]: starting mds.c at 2024-05-14T17:19:19.294 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: ignoring --setuser ceph since I am not root 2024-05-14T17:19:19.294 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: ignoring --setgroup ceph since I am not root 2024-05-14T17:19:19.294 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: starting mds.f at 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: ignoring --setuser ceph since I am not root 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: ignoring --setgroup ceph since I am not root 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:19:18 smithi193 ceph-mds[67694]: -- 172.21.15.193:0/1631086700 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eea39f7a40 con 0x55eea39cac00 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: starting mds.i at 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l[66763]: ignoring --setuser ceph since I am not root 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l[66763]: ignoring --setgroup ceph since I am not root 2024-05-14T17:19:19.295 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:19:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l[66763]: starting mds.l at 2024-05-14T17:19:19.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T17:19:19.889 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:21.107 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: pgmap v497: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6836/62266559,v1:172.21.15.150:6838/62266559] up:boot 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.150:6837/3643527870,v1:172.21.15.150:6839/3643527870] up:boot 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6833/1164407672,v1:172.21.15.193:6834/1164407672] up:boot 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6832/3308048073,v1:172.21.15.193:6835/3308048073] up:boot 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6836/405925654,v1:172.21.15.193:6838/405925654] up:boot 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: mds.? [v2:172.21.15.193:6837/2242088256,v1:172.21.15.193:6839/2242088256] up:boot 2024-05-14T17:19:21.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T17:19:21.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:19:21.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:19:21.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:19:21.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:19:21.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:19:21.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:20 smithi099 ceph-mon[31851]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:19:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: pgmap v497: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-05-14T17:19:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6836/62266559,v1:172.21.15.150:6838/62266559] up:boot 2024-05-14T17:19:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.150:6837/3643527870,v1:172.21.15.150:6839/3643527870] up:boot 2024-05-14T17:19:21.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6833/1164407672,v1:172.21.15.193:6834/1164407672] up:boot 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6832/3308048073,v1:172.21.15.193:6835/3308048073] up:boot 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6836/405925654,v1:172.21.15.193:6838/405925654] up:boot 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: mds.? [v2:172.21.15.193:6837/2242088256,v1:172.21.15.193:6839/2242088256] up:boot 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:19:21.204 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:19:21.205 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:20 smithi150 ceph-mon[36204]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:19:21.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: pgmap v497: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-05-14T17:19:21.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6836/62266559,v1:172.21.15.150:6838/62266559] up:boot 2024-05-14T17:19:21.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.150:6837/3643527870,v1:172.21.15.150:6839/3643527870] up:boot 2024-05-14T17:19:21.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6833/1164407672,v1:172.21.15.193:6834/1164407672] up:boot 2024-05-14T17:19:21.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6832/3308048073,v1:172.21.15.193:6835/3308048073] up:boot 2024-05-14T17:19:21.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6836/405925654,v1:172.21.15.193:6838/405925654] up:boot 2024-05-14T17:19:21.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: mds.? [v2:172.21.15.193:6837/2242088256,v1:172.21.15.193:6839/2242088256] up:boot 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:19:21.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:20 smithi193 ceph-mon[35962]: from='mgr.24710 172.21.15.99:0/4242478530' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T17:19:21.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 -- 172.21.15.99:0/2050330272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806bab0 msgr2=0x7f121806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:21.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 --2- 172.21.15.99:0/2050330272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806bab0 0x7f121806bf30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1200009990 tx=0x7f1200009ca0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 -- 172.21.15.99:0/2050330272 shutdown_connections 2024-05-14T17:19:21.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 --2- 172.21.15.99:0/2050330272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121806c470 0x7f12181103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 --2- 172.21.15.99:0/2050330272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12181095a0 0x7f1218109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 --2- 172.21.15.99:0/2050330272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806bab0 0x7f121806bf30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 -- 172.21.15.99:0/2050330272 >> 172.21.15.99:0/2050330272 conn(0x7f12180793b0 msgr2=0x7f12180797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:21.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 -- 172.21.15.99:0/2050330272 shutdown_connections 2024-05-14T17:19:21.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.477+0000 7f121d56d700 1 -- 172.21.15.99:0/2050330272 wait complete. 2024-05-14T17:19:21.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 Processor -- start 2024-05-14T17:19:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 -- start start 2024-05-14T17:19:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121806bab0 0x7f121819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 0x7f121819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12181095a0 0x7f1218199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1218113730 con 0x7f121806c470 2024-05-14T17:19:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f12181135b0 con 0x7f12181095a0 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f121d56d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12181138b0 con 0x7f121806bab0 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f12167fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 0x7f121819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f12167fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 0x7f121819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53428/0 (socket says 172.21.15.99:53428) 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f12167fc700 1 -- 172.21.15.99:0/2581953099 learned_addr learned my addr 172.21.15.99:0/2581953099 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f1216ffd700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121806bab0 0x7f121819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f12177fe700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12181095a0 0x7f1218199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f12167fc700 1 -- 172.21.15.99:0/2581953099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121806bab0 msgr2=0x7f121819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.478+0000 7f12167fc700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121806bab0 0x7f121819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f12167fc700 1 -- 172.21.15.99:0/2581953099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12181095a0 msgr2=0x7f1218199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f12167fc700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12181095a0 0x7f1218199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f12167fc700 1 -- 172.21.15.99:0/2581953099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121819a1d0 con 0x7f121806c470 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f1216ffd700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121806bab0 0x7f121819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f12167fc700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 0x7f121819fdf0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1200000c00 tx=0x7f12000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200027070 con 0x7f121806c470 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121819a460 con 0x7f121806c470 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12000041d0 con 0x7f121806c470 2024-05-14T17:19:21.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120000f460 con 0x7f121806c470 2024-05-14T17:19:21.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.479+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181a6780 con 0x7f121806c470 2024-05-14T17:19:21.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.480+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121806a230 con 0x7f121806c470 2024-05-14T17:19:21.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.481+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f120002a070 con 0x7f121806c470 2024-05-14T17:19:21.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.482+0000 7f120ffff700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f12040b29f0 0x7f12040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:21.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.482+0000 7f1216ffd700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f12040b29f0 0x7f12040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:21.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.483+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8735+0+0 (secure 0 0 0) 0x7f12000e3b30 con 0x7f121806c470 2024-05-14T17:19:21.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.483+0000 7f1216ffd700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f12040b29f0 0x7f12040b4eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f121819abb0 tx=0x7f1210009040 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:21.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.486+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1200031070 con 0x7f121806c470 2024-05-14T17:19:21.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.737+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f121819afa0 con 0x7f121806c470 2024-05-14T17:19:21.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.851+0000 7f120ffff700 1 -- 172.21.15.99:0/2581953099 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f1200013070 con 0x7f121806c470 2024-05-14T17:19:21.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f12040b29f0 msgr2=0x7f12040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:21.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f12040b29f0 0x7f12040b4eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f121819abb0 tx=0x7f1210009040 comp rx=0 tx=0).stop 2024-05-14T17:19:21.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 msgr2=0x7f121819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:21.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 0x7f121819fdf0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1200000c00 tx=0x7f12000047b0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 shutdown_connections 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f12040b29f0 0x7f12040b4eb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12181095a0 0x7f1218199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f121806bab0 0x7f121819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 --2- 172.21.15.99:0/2581953099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121806c470 0x7f121819fdf0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 >> 172.21.15.99:0/2581953099 conn(0x7f12180793b0 msgr2=0x7f12181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 shutdown_connections 2024-05-14T17:19:21.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:21.853+0000 7f121d56d700 1 -- 172.21.15.99:0/2581953099 wait complete. 2024-05-14T17:19:22.014 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:21 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2581953099' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:19:22.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:21 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2581953099' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:19:22.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:21 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2581953099' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:19:22.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T17:19:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:23.137 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:22 smithi099 ceph-mon[31851]: pgmap v498: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-05-14T17:19:23.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:22 smithi099 ceph-mon[31851]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:19:23.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:22 smithi099 ceph-mon[31851]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:19:23.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:22 smithi099 ceph-mon[31851]: Cluster is now healthy 2024-05-14T17:19:23.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:22 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2581953099' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:19:23.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:22 smithi099 ceph-mon[31851]: fsmap 12 up:standby 2024-05-14T17:19:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:22 smithi150 ceph-mon[36204]: pgmap v498: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-05-14T17:19:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:22 smithi150 ceph-mon[36204]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:19:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:22 smithi150 ceph-mon[36204]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:19:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:22 smithi150 ceph-mon[36204]: Cluster is now healthy 2024-05-14T17:19:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:22 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2581953099' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:19:23.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:22 smithi150 ceph-mon[36204]: fsmap 12 up:standby 2024-05-14T17:19:23.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:22 smithi193 ceph-mon[35962]: pgmap v498: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.6 KiB/s wr, 4 op/s 2024-05-14T17:19:23.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:22 smithi193 ceph-mon[35962]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:19:23.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:22 smithi193 ceph-mon[35962]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:19:23.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:22 smithi193 ceph-mon[35962]: Cluster is now healthy 2024-05-14T17:19:23.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:22 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2581953099' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:19:23.293 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:22 smithi193 ceph-mon[35962]: fsmap 12 up:standby 2024-05-14T17:19:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2585167050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 msgr2=0x7f4f08110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2585167050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08110470 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b0a0 tx=0x7f4f0000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2585167050 shutdown_connections 2024-05-14T17:19:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2585167050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08110470 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2585167050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f08108500 0x7f4f081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2585167050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f081035f0 0x7f4f08103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2585167050 >> 172.21.15.99:0/2585167050 conn(0x7f4f0807a6a0 msgr2=0x7f4f0807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.459+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2585167050 shutdown_connections 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.460+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2585167050 wait complete. 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.460+0000 7f4f0d3cb700 1 Processor -- start 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.460+0000 7f4f0d3cb700 1 -- start start 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f0d3cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f081035f0 0x7f4f08075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:24.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f0d3cb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f0d3cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f08108500 0x7f4f080762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f0d3cb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081137e0 con 0x7f4f08103f90 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f0d3cb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113660 con 0x7f4f08108500 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f0d3cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08113960 con 0x7f4f081035f0 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53454/0 (socket says 172.21.15.99:53454) 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 -- 172.21.15.99:0/2046962817 learned_addr learned my addr 172.21.15.99:0/2046962817 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f06ffd700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f081035f0 0x7f4f08075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f077fe700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f08108500 0x7f4f080762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 -- 172.21.15.99:0/2046962817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f081035f0 msgr2=0x7f4f08075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f081035f0 0x7f4f08075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 -- 172.21.15.99:0/2046962817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f08108500 msgr2=0x7f4f080762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f08108500 0x7f4f080762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 -- 172.21.15.99:0/2046962817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a8100 con 0x7f4f08103f90 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4f067fc700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08075d80 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c9b0 tx=0x7f4efc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.461+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc007b00 con 0x7f4f08103f90 2024-05-14T17:19:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.462+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a83f0 con 0x7f4f08103f90 2024-05-14T17:19:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.462+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4efc00ce50 con 0x7f4f08103f90 2024-05-14T17:19:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.462+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc018660 con 0x7f4f08103f90 2024-05-14T17:19:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.462+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a8930 con 0x7f4f08103f90 2024-05-14T17:19:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.463+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0810bb30 con 0x7f4f08103f90 2024-05-14T17:19:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.464+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4efc00f450 con 0x7f4f08103f90 2024-05-14T17:19:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.465+0000 7f4eeffff700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4ef00b29f0 0x7f4ef00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.465+0000 7f4f06ffd700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4ef00b29f0 0x7f4ef00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.465+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8735+0+0 (secure 0 0 0) 0x7f4efc0e2610 con 0x7f4f08103f90 2024-05-14T17:19:24.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.465+0000 7f4f06ffd700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4ef00b29f0 0x7f4ef00b4eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005950 tx=0x7f4ef800a300 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:24.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.469+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4efc014030 con 0x7f4f08103f90 2024-05-14T17:19:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.670+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4f08076d70 con 0x7f4f08103f90 2024-05-14T17:19:24.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.850+0000 7f4eeffff700 1 -- 172.21.15.99:0/2046962817 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7f4efc0a9f20 con 0x7f4f08103f90 2024-05-14T17:19:24.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4ef00b29f0 msgr2=0x7f4ef00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:24.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4ef00b29f0 0x7f4ef00b4eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005950 tx=0x7f4ef800a300 comp rx=0 tx=0).stop 2024-05-14T17:19:24.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 msgr2=0x7f4f08075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:24.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08075d80 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c9b0 tx=0x7f4efc00ccc0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 shutdown_connections 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7f4ef00b29f0 0x7f4ef00b4eb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f08108500 0x7f4f080762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f081035f0 0x7f4f08075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 --2- 172.21.15.99:0/2046962817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f08103f90 0x7f4f08075d80 secure :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c9b0 tx=0x7f4efc00ccc0 comp rx=0 tx=0).stop 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 >> 172.21.15.99:0/2046962817 conn(0x7f4f0807a6a0 msgr2=0x7f4f081016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.854+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 shutdown_connections 2024-05-14T17:19:24.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:24.855+0000 7f4f0d3cb700 1 -- 172.21.15.99:0/2046962817 wait complete. 2024-05-14T17:19:24.856 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-05-14T17:19:25.182 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:24 smithi099 ceph-mon[31851]: pgmap v499: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-05-14T17:19:25.182 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:24 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2046962817' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:19:25.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:24 smithi150 ceph-mon[36204]: pgmap v499: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-05-14T17:19:25.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:24 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2046962817' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:19:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:24 smithi193 ceph-mon[35962]: pgmap v499: 97 pgs: 97 active+clean; 393 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-05-14T17:19:25.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:24 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2046962817' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:19:25.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid 2a006d80-1212-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T17:19:25.752 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:19:25.952 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:25 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/2046962817' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:19:25.953 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:25 smithi150 ceph-mon[36204]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:19:26.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:25 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/2046962817' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:19:26.162 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:25 smithi099 ceph-mon[31851]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:19:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:25 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/2046962817' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:19:26.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:25 smithi193 ceph-mon[35962]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:19:27.130 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:26 smithi099 ceph-mon[31851]: pgmap v501: 65 pgs: 65 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:19:27.202 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:26 smithi150 ceph-mon[36204]: pgmap v501: 65 pgs: 65 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:19:27.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:26 smithi193 ceph-mon[35962]: pgmap v501: 65 pgs: 65 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:19:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 -- 172.21.15.99:0/4021613666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b01041d0 msgr2=0x7fa9b01081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 --2- 172.21.15.99:0/4021613666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b01041d0 0x7fa9b01081e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00099c0 tx=0x7fa9a0009cd0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 -- 172.21.15.99:0/4021613666 shutdown_connections 2024-05-14T17:19:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 --2- 172.21.15.99:0/4021613666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b0108910 0x7fa9b0110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 --2- 172.21.15.99:0/4021613666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b0103820 0x7fa9b0103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 --2- 172.21.15.99:0/4021613666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b01041d0 0x7fa9b01081e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 -- 172.21.15.99:0/4021613666 >> 172.21.15.99:0/4021613666 conn(0x7fa9b007a6b0 msgr2=0x7fa9b007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 -- 172.21.15.99:0/4021613666 shutdown_connections 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.378+0000 7fa9b850b700 1 -- 172.21.15.99:0/4021613666 wait complete. 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 Processor -- start 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 -- start start 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b0103820 0x7fa9b0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:27.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b01041d0 0x7fa9b0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 0x7fa9b00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0113750 con 0x7fa9b0108910 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01135d0 con 0x7fa9b01041d0 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b850b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01138d0 con 0x7fa9b0103820 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b6aa8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 0x7fa9b00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b6aa8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 0x7fa9b00762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33174/0 (socket says 172.21.15.99:33174) 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.379+0000 7fa9b6aa8700 1 -- 172.21.15.99:0/3762807089 learned_addr learned my addr 172.21.15.99:0/3762807089 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-14T17:19:27.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b62a7700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b0103820 0x7fa9b0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b5aa6700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b01041d0 0x7fa9b0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b6aa8700 1 -- 172.21.15.99:0/3762807089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b0103820 msgr2=0x7fa9b0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b6aa8700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b0103820 0x7fa9b0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b6aa8700 1 -- 172.21.15.99:0/3762807089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b01041d0 msgr2=0x7fa9b0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b6aa8700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b01041d0 0x7fa9b0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b6aa8700 1 -- 172.21.15.99:0/3762807089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b01a8070 con 0x7fa9b0108910 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b5aa6700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b01041d0 0x7fa9b0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:19:27.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b6aa8700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 0x7fa9b00762c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800b970 tx=0x7fa9a800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8019070 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a8014440 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01a8360 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800ce60 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.380+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a8850 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.381+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b006a230 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.383+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa9a801c070 con 0x7fa9b0108910 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.383+0000 7fa9a77fe700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa99c0b29f0 0x7fa99c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:19:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.383+0000 7fa9b62a7700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa99c0b29f0 0x7fa99c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:19:27.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.383+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8210+0+0 (secure 0 0 0) 0x7fa9a80e1d20 con 0x7fa9b0108910 2024-05-14T17:19:27.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.384+0000 7fa9b62a7700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa99c0b29f0 0x7fa99c0b4eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac005ea0 tx=0x7fa9ac005e30 comp rx=0 tx=0).ready entity=mgr.24710 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:19:27.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.386+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9a80a9940 con 0x7fa9b0108910 2024-05-14T17:19:27.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.590+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9b01a8c70 con 0x7fa9b0108910 2024-05-14T17:19:27.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.875+0000 7fa9a77fe700 1 -- 172.21.15.99:0/3762807089 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7fa9a80a9240 con 0x7fa9b0108910 2024-05-14T17:19:27.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa99c0b29f0 msgr2=0x7fa99c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:27.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa99c0b29f0 0x7fa99c0b4eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac005ea0 tx=0x7fa9ac005e30 comp rx=0 tx=0).stop 2024-05-14T17:19:27.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 msgr2=0x7fa9b00762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:19:27.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 0x7fa9b00762c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800b970 tx=0x7fa9a800bd30 comp rx=0 tx=0).stop 2024-05-14T17:19:27.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 shutdown_connections 2024-05-14T17:19:27.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:6800/2715605784,v1:172.21.15.99:6801/2715605784] conn(0x7fa99c0b29f0 0x7fa99c0b4eb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b0108910 0x7fa9b00762c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b0103820 0x7fa9b0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 --2- 172.21.15.99:0/3762807089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b01041d0 0x7fa9b0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:19:27.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 >> 172.21.15.99:0/3762807089 conn(0x7fa9b007a6b0 msgr2=0x7fa9b0101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:19:27.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 shutdown_connections 2024-05-14T17:19:27.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-14T17:19:27.878+0000 7fa9b850b700 1 -- 172.21.15.99:0/3762807089 wait complete. 2024-05-14T17:19:27.880 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-05-14T17:19:28.173 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:27 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3762807089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:19:28.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:27 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3762807089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:19:28.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:27 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3762807089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:19:28.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:19:28.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:19:28.541 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:19:28.569 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-14T17:19:28.569 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:19:28.585 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:19:28.603 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:19:28.623 INFO:tasks.cephadm:Stopping all daemons... 2024-05-14T17:19:28.624 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-14T17:19:28.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a 2024-05-14T17:19:28.764 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:28 smithi099 systemd[1]: Stopping Ceph mon.a for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:28 smithi150 ceph-mon[36204]: pgmap v502: 65 pgs: 65 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:19:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:28 smithi150 ceph-mon[36204]: from='client.? 172.21.15.99:0/3762807089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:19:29.203 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:28 smithi150 ceph-mon[36204]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:19:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:28 smithi099 ceph-mon[31851]: pgmap v502: 65 pgs: 65 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:19:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:28 smithi099 ceph-mon[31851]: from='client.? 172.21.15.99:0/3762807089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:19:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:28 smithi099 ceph-mon[31851]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:19:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a[31847]: 2024-05-14T17:19:29.011+0000 7fd079fc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a[31847]: 2024-05-14T17:19:29.011+0000 7fd079fc9700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-14T17:19:29.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 podman[106176]: 2024-05-14 17:19:29.094343816 +0000 UTC m=+0.273565555 container died 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:28 smithi193 ceph-mon[35962]: pgmap v502: 65 pgs: 65 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:19:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:28 smithi193 ceph-mon[35962]: from='client.? 172.21.15.99:0/3762807089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:19:29.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:28 smithi193 ceph-mon[35962]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:19:29.703 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 podman[106176]: 2024-05-14 17:19:29.503196304 +0000 UTC m=+0.682418037 container cleanup 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-14T17:19:29.703 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 bash[106176]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a 2024-05-14T17:19:29.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service' 2024-05-14T17:19:29.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 podman[106188]: 2024-05-14 17:19:29.704467768 +0000 UTC m=+0.607993588 container remove 6f2cc53d7976cac732a487ab8aed26c4ee4c00ea0af5bc5baa960c4521244119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-14T17:19:29.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T17:19:29.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 systemd[1]: Stopped Ceph mon.a for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:19:29.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:May 14 17:19:29 smithi099 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.a.service: Consumed 21.846s CPU time. 2024-05-14T17:19:29.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:29.996 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-14T17:19:29.996 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-14T17:19:29.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.b 2024-05-14T17:19:30.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:30 smithi150 systemd[1]: Stopping Ceph mon.b for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:30.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-b[36200]: 2024-05-14T17:19:30.322+0000 7f779f173700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:30.703 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-b[36200]: 2024-05-14T17:19:30.322+0000 7f779f173700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-14T17:19:30.704 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:30 smithi150 podman[73738]: 2024-05-14 17:19:30.507980627 +0000 UTC m=+0.319978045 container died a6945f6732f79ec08504d60eb9b3b82d5191b79812be8de5da8a0f5d71ab8fb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-b, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-14T17:19:31.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:30 smithi150 podman[73738]: 2024-05-14 17:19:30.816002344 +0000 UTC m=+0.627999758 container cleanup a6945f6732f79ec08504d60eb9b3b82d5191b79812be8de5da8a0f5d71ab8fb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-b, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-14T17:19:31.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:30 smithi150 bash[73738]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-b 2024-05-14T17:19:31.286 INFO:journalctl@ceph.mon.b.smithi150.stdout:May 14 17:19:31 smithi150 podman[73750]: 2024-05-14 17:19:31.033868379 +0000 UTC m=+0.525707165 container remove a6945f6732f79ec08504d60eb9b3b82d5191b79812be8de5da8a0f5d71ab8fb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:31.290 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.b.service' 2024-05-14T17:19:31.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:31.335 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-14T17:19:31.335 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-14T17:19:31.335 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.c 2024-05-14T17:19:31.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:31 smithi193 systemd[1]: Stopping Ceph mon.c for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:32.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c[35958]: 2024-05-14T17:19:31.637+0000 7f94bf65e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:32.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c[35958]: 2024-05-14T17:19:31.637+0000 7f94bf65e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-14T17:19:32.042 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:31 smithi193 podman[75512]: 2024-05-14 17:19:31.802958752 +0000 UTC m=+0.265598010 container died 2e8dd14d2223cb36910c68b333735cb737dfb38b021d3d88e858045792737397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:32.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:32 smithi193 podman[75512]: 2024-05-14 17:19:32.180907577 +0000 UTC m=+0.643546836 container cleanup 2e8dd14d2223cb36910c68b333735cb737dfb38b021d3d88e858045792737397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:32.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 17:19:32 smithi193 bash[75512]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mon-c 2024-05-14T17:19:32.628 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mon.c.service' 2024-05-14T17:19:32.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:32.674 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-14T17:19:32.675 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-14T17:19:32.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x 2024-05-14T17:19:33.014 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:19:32 smithi099 systemd[1]: Stopping Ceph mgr.x for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:33.456 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:19:33 smithi099 podman[106297]: 2024-05-14 17:19:33.046702573 +0000 UTC m=+0.172520514 container died f9791659271ca4a669423acb1ea59707e91968265ca491f2798ef6bc4f6a8d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-14T17:19:33.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:19:33 smithi099 podman[106297]: 2024-05-14 17:19:33.457935977 +0000 UTC m=+0.583753916 container cleanup f9791659271ca4a669423acb1ea59707e91968265ca491f2798ef6bc4f6a8d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:33.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:19:33 smithi099 bash[106297]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x 2024-05-14T17:19:33.764 INFO:journalctl@ceph.mgr.x.smithi099.stdout:May 14 17:19:33 smithi099 podman[106311]: 2024-05-14 17:19:33.651348785 +0000 UTC m=+0.600500794 container remove f9791659271ca4a669423acb1ea59707e91968265ca491f2798ef6bc4f6a8d27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T17:19:33.896 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-14T17:19:33.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:33.934 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-14T17:19:33.934 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-14T17:19:33.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.y 2024-05-14T17:19:34.248 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:19:33 smithi150 systemd[1]: Stopping Ceph mgr.y for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:34.529 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:19:34 smithi150 podman[73860]: 2024-05-14 17:19:34.24935079 +0000 UTC m=+0.120403262 container died 6b35e6d20c5642d2fb53228bddfb5abd71dc4740d4d00ccc724707884a839a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T17:19:34.905 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:19:34 smithi150 podman[73860]: 2024-05-14 17:19:34.629430152 +0000 UTC m=+0.500482621 container cleanup 6b35e6d20c5642d2fb53228bddfb5abd71dc4740d4d00ccc724707884a839a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:34.905 INFO:journalctl@ceph.mgr.y.smithi150.stdout:May 14 17:19:34 smithi150 bash[73860]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-y 2024-05-14T17:19:35.055 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-14T17:19:35.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:35.088 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-14T17:19:35.088 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-14T17:19:35.089 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.z 2024-05-14T17:19:35.542 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:19:35 smithi193 systemd[1]: Stopping Ceph mgr.z for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:35.844 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:19:35 smithi193 podman[75634]: 2024-05-14 17:19:35.558710051 +0000 UTC m=+0.271367091 container died b02a4c2b190912f25da7ab386f87baa4b2006e3fc1471b085378f8dc2fa31b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-14T17:19:36.186 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:19:35 smithi193 podman[75634]: 2024-05-14 17:19:35.927678586 +0000 UTC m=+0.640335623 container cleanup b02a4c2b190912f25da7ab386f87baa4b2006e3fc1471b085378f8dc2fa31b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-14T17:19:36.186 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 17:19:35 smithi193 bash[75634]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mgr-z 2024-05-14T17:19:36.367 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-14T17:19:36.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:36.400 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-14T17:19:36.400 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-14T17:19:36.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.0 2024-05-14T17:19:36.711 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:36 smithi099 systemd[1]: Stopping Ceph osd.0 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:37.014 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0[44343]: 2024-05-14T17:19:36.711+0000 7f84b7bd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:37.014 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0[44343]: 2024-05-14T17:19:36.711+0000 7f84b7bd7700 -1 osd.0 108 *** Got signal Terminated *** 2024-05-14T17:19:37.014 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0[44343]: 2024-05-14T17:19:36.711+0000 7f84b7bd7700 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:42.470 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:42 smithi099 podman[106419]: 2024-05-14 17:19:42.157102428 +0000 UTC m=+5.562710005 container died 3034211c3be0ec0a32d8f3c354112d559d23cf2d0104fc634264415434df0a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS) 2024-05-14T17:19:42.764 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:42 smithi099 podman[106419]: 2024-05-14 17:19:42.679000229 +0000 UTC m=+6.084607808 container cleanup 3034211c3be0ec0a32d8f3c354112d559d23cf2d0104fc634264415434df0a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-14T17:19:42.764 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:42 smithi099 bash[106419]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0 2024-05-14T17:19:43.264 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:42 smithi099 podman[106436]: 2024-05-14 17:19:42.863811741 +0000 UTC m=+0.705456081 container remove 3034211c3be0ec0a32d8f3c354112d559d23cf2d0104fc634264415434df0a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T17:19:43.540 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:43 smithi099 podman[106523]: 2024-05-14 17:19:43.202950387 +0000 UTC m=+0.017050305 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:43.541 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:43 smithi099 podman[106523]: 2024-05-14 17:19:43.331469226 +0000 UTC m=+0.145569144 container create d32166802cd7c32ceca854caafe4076c818fbe603c42e8d3d683f259fd479931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-14T17:19:43.835 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:43 smithi099 podman[106523]: 2024-05-14 17:19:43.54208471 +0000 UTC m=+0.356184627 container init d32166802cd7c32ceca854caafe4076c818fbe603c42e8d3d683f259fd479931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T17:19:43.835 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:43 smithi099 podman[106523]: 2024-05-14 17:19:43.54562901 +0000 UTC m=+0.359728931 container start d32166802cd7c32ceca854caafe4076c818fbe603c42e8d3d683f259fd479931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:19:43.835 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:43 smithi099 podman[106523]: 2024-05-14 17:19:43.650756663 +0000 UTC m=+0.464856582 container attach d32166802cd7c32ceca854caafe4076c818fbe603c42e8d3d683f259fd479931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:44.253 INFO:journalctl@ceph.osd.0.smithi099.stdout:May 14 17:19:43 smithi099 podman[106523]: 2024-05-14 17:19:43.836811468 +0000 UTC m=+0.650911394 container died d32166802cd7c32ceca854caafe4076c818fbe603c42e8d3d683f259fd479931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True) 2024-05-14T17:19:44.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.0.service' 2024-05-14T17:19:44.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:44.575 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-14T17:19:44.575 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-14T17:19:44.576 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.1 2024-05-14T17:19:44.876 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:44 smithi150 systemd[1]: Stopping Ceph osd.1 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:45.203 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1[40776]: 2024-05-14T17:19:44.876+0000 7f24a092c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:45.203 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1[40776]: 2024-05-14T17:19:44.876+0000 7f24a092c700 -1 osd.1 108 *** Got signal Terminated *** 2024-05-14T17:19:45.203 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1[40776]: 2024-05-14T17:19:44.876+0000 7f24a092c700 -1 osd.1 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:50.686 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:50 smithi150 podman[73995]: 2024-05-14 17:19:50.397041939 +0000 UTC m=+5.613754094 container died 508e949825b88cabe7f895b7fee9a15c9871f44e2046fcaa256adcb3c5851903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T17:19:50.953 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:50 smithi150 podman[73995]: 2024-05-14 17:19:50.77793523 +0000 UTC m=+5.994647384 container cleanup 508e949825b88cabe7f895b7fee9a15c9871f44e2046fcaa256adcb3c5851903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-14T17:19:50.953 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:50 smithi150 bash[73995]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1 2024-05-14T17:19:51.395 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74014]: 2024-05-14 17:19:51.063058692 +0000 UTC m=+0.663617019 container remove 508e949825b88cabe7f895b7fee9a15c9871f44e2046fcaa256adcb3c5851903 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-14T17:19:51.703 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74108]: 2024-05-14 17:19:51.298940698 +0000 UTC m=+0.017230699 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:51.703 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74108]: 2024-05-14 17:19:51.422344348 +0000 UTC m=+0.140634366 container create 64c87f6016c01c73f8b88785bc5edee8d74eabea6f465502ba646027cd9808f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-14T17:19:51.703 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74108]: 2024-05-14 17:19:51.641349484 +0000 UTC m=+0.359639471 container init 64c87f6016c01c73f8b88785bc5edee8d74eabea6f465502ba646027cd9808f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:19:51.703 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74108]: 2024-05-14 17:19:51.644497568 +0000 UTC m=+0.362787573 container start 64c87f6016c01c73f8b88785bc5edee8d74eabea6f465502ba646027cd9808f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-14T17:19:52.185 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74108]: 2024-05-14 17:19:51.724692933 +0000 UTC m=+0.442982953 container attach 64c87f6016c01c73f8b88785bc5edee8d74eabea6f465502ba646027cd9808f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:52.185 INFO:journalctl@ceph.osd.1.smithi150.stdout:May 14 17:19:51 smithi150 podman[74108]: 2024-05-14 17:19:51.951365571 +0000 UTC m=+0.669655568 container died 64c87f6016c01c73f8b88785bc5edee8d74eabea6f465502ba646027cd9808f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-14T17:19:52.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.1.service' 2024-05-14T17:19:52.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:52.547 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-14T17:19:52.548 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-14T17:19:52.548 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.2 2024-05-14T17:19:52.855 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:52 smithi193 systemd[1]: Stopping Ceph osd.2 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:19:53.224 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2[41088]: 2024-05-14T17:19:52.855+0000 7f17cc701700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:53.225 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2[41088]: 2024-05-14T17:19:52.855+0000 7f17cc701700 -1 osd.2 108 *** Got signal Terminated *** 2024-05-14T17:19:53.225 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2[41088]: 2024-05-14T17:19:52.855+0000 7f17cc701700 -1 osd.2 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:58.680 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:58 smithi193 podman[75759]: 2024-05-14 17:19:58.360211209 +0000 UTC m=+5.616128815 container died 3bbcaab0f8894f9b3af6be1e8a6d30a9e2dcdd2913ae28b2ff876a7fd04aed92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:59.042 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:58 smithi193 podman[75759]: 2024-05-14 17:19:58.681373961 +0000 UTC m=+5.937291567 container cleanup 3bbcaab0f8894f9b3af6be1e8a6d30a9e2dcdd2913ae28b2ff876a7fd04aed92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T17:19:59.042 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:58 smithi193 bash[75759]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2 2024-05-14T17:19:59.042 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:58 smithi193 podman[75788]: 2024-05-14 17:19:58.890822719 +0000 UTC m=+0.533273916 container remove 3bbcaab0f8894f9b3af6be1e8a6d30a9e2dcdd2913ae28b2ff876a7fd04aed92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:59.617 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:59 smithi193 podman[75877]: 2024-05-14 17:19:59.220178939 +0000 UTC m=+0.017269147 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:59.618 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:59 smithi193 podman[75877]: 2024-05-14 17:19:59.325074114 +0000 UTC m=+0.122164295 container create a93df0554f52fbff71eaa30435d74d813174c78e84fc955da8207bbf2717881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:59.618 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:59 smithi193 podman[75877]: 2024-05-14 17:19:59.527343842 +0000 UTC m=+0.324434024 container init a93df0554f52fbff71eaa30435d74d813174c78e84fc955da8207bbf2717881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:59.618 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:59 smithi193 podman[75877]: 2024-05-14 17:19:59.530561767 +0000 UTC m=+0.327651960 container start a93df0554f52fbff71eaa30435d74d813174c78e84fc955da8207bbf2717881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-14T17:19:59.618 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:59 smithi193 podman[75877]: 2024-05-14 17:19:59.619059621 +0000 UTC m=+0.416149814 container attach a93df0554f52fbff71eaa30435d74d813174c78e84fc955da8207bbf2717881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-05-14T17:20:00.042 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 17:19:59 smithi193 podman[75877]: 2024-05-14 17:19:59.823357543 +0000 UTC m=+0.620447733 container died a93df0554f52fbff71eaa30435d74d813174c78e84fc955da8207bbf2717881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:00.453 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.2.service' 2024-05-14T17:20:00.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:00.492 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-14T17:20:00.492 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-14T17:20:00.492 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.3 2024-05-14T17:20:00.780 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:00 smithi099 systemd[1]: Stopping Ceph osd.3 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:01.264 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T17:20:00.780+0000 7fdb41efc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:01.264 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T17:20:00.780+0000 7fdb41efc700 -1 osd.3 108 *** Got signal Terminated *** 2024-05-14T17:20:01.264 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:00 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T17:20:00.780+0000 7fdb41efc700 -1 osd.3 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:03.514 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:03 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T17:20:03.068+0000 7fdb3e4f0700 -1 osd.3 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:40.201264+0000 front 2024-05-14T17:19:40.201285+0000 (oldest deadline 2024-05-14T17:20:02.501493+0000) 2024-05-14T17:20:03.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:03 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:03.234+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:04.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:03.646+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:04.264 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:04 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T17:20:04.081+0000 7fdb3e4f0700 -1 osd.3 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:40.201264+0000 front 2024-05-14T17:19:40.201285+0000 (oldest deadline 2024-05-14T17:20:02.501493+0000) 2024-05-14T17:20:04.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:03 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:03.940+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:04.265 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:04 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:04.142+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:04.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:04 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:04.218+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:04.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:04.614+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:05.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:04 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:04.962+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:05.265 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:05 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3[51255]: 2024-05-14T17:20:05.093+0000 7fdb3e4f0700 -1 osd.3 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:40.201264+0000 front 2024-05-14T17:19:40.201285+0000 (oldest deadline 2024-05-14T17:20:02.501493+0000) 2024-05-14T17:20:05.265 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:05 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:05.107+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:05.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:04.978+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:05.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:05 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:05.207+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:05.944 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:05.578+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:06.087 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:05 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:05.940+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:06.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:05.945+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:06.376 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:06 smithi099 podman[106650]: 2024-05-14 17:20:06.259462378 +0000 UTC m=+5.571566649 container died 470c8c9a8bc46dc863f969b65061c131da5ec9b3b447557c4ac91addac277ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T17:20:06.376 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:06 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:06.088+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:06.452 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:06 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:06.172+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:06.928 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:06.601+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:06.938 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:06 smithi099 podman[106650]: 2024-05-14 17:20:06.632099122 +0000 UTC m=+5.944203385 container cleanup 470c8c9a8bc46dc863f969b65061c131da5ec9b3b447557c4ac91addac277ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T17:20:06.938 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:06 smithi099 bash[106650]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3 2024-05-14T17:20:06.938 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:06 smithi099 podman[106665]: 2024-05-14 17:20:06.841711208 +0000 UTC m=+0.579763322 container remove 470c8c9a8bc46dc863f969b65061c131da5ec9b3b447557c4ac91addac277ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:07.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:07 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:07.082+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:07.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:06 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:06.939+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:07.265 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:07 smithi099 podman[106755]: 2024-05-14 17:20:07.159308131 +0000 UTC m=+0.099089486 container create 97087e67a6b2baeffff7696ed3026a507780b0bc52edad481ed47d8a795d263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:07.265 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:07 smithi099 podman[106755]: 2024-05-14 17:20:07.077320059 +0000 UTC m=+0.017101438 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:07.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:06.929+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:07.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:07 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:07.161+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:07.665 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:07 smithi099 podman[106755]: 2024-05-14 17:20:07.370031693 +0000 UTC m=+0.309813067 container init 97087e67a6b2baeffff7696ed3026a507780b0bc52edad481ed47d8a795d263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-14T17:20:07.665 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:07 smithi099 podman[106755]: 2024-05-14 17:20:07.373238964 +0000 UTC m=+0.313020330 container start 97087e67a6b2baeffff7696ed3026a507780b0bc52edad481ed47d8a795d263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514) 2024-05-14T17:20:07.665 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:07 smithi099 podman[106755]: 2024-05-14 17:20:07.461599829 +0000 UTC m=+0.401381214 container attach 97087e67a6b2baeffff7696ed3026a507780b0bc52edad481ed47d8a795d263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:07.666 INFO:journalctl@ceph.osd.3.smithi099.stdout:May 14 17:20:07 smithi099 podman[106755]: 2024-05-14 17:20:07.666183055 +0000 UTC m=+0.605964425 container died 97087e67a6b2baeffff7696ed3026a507780b0bc52edad481ed47d8a795d263d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-14T17:20:07.909 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:07.632+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:07.955 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:07 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:07.896+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:08.205 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:08 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:08.087+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:08.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.3.service' 2024-05-14T17:20:08.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:08.262 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-14T17:20:08.262 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-14T17:20:08.262 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.4 2024-05-14T17:20:08.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:07.910+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:08.453 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:08 smithi150 systemd[1]: Stopping Ceph osd.4 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:08.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:08 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:08.141+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:08.927 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:08 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:08.582+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:08.952 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:08 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4[47246]: 2024-05-14T17:20:08.554+0000 7f7840b8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:08.953 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:08 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4[47246]: 2024-05-14T17:20:08.554+0000 7f7840b8c700 -1 osd.4 108 *** Got signal Terminated *** 2024-05-14T17:20:08.953 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:08 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4[47246]: 2024-05-14T17:20:08.554+0000 7f7840b8c700 -1 osd.4 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:09.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:09 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:09.086+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:09.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:08 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:08.946+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:09.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:08 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:08.927+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:09.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:09 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:09.185+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:09.895 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:09 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:09.628+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:10.152 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:09 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:09.896+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:10.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:10 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:10.072+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:10.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:09 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:09.908+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:10.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:10 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:10.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:10.921 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:10 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:10.598+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:11.210 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:10 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:10.922+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:11.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:10 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:10.923+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:11.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:11 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:11.035+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:11.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:11 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:11.035+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:11.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:11 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:11.211+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:11.702 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:11 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:11.204+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:11.920 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:11 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:11.617+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:12.249 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:11 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:11.921+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:12.249 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:11 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:11.921+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:12.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:11 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:11.933+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:12.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:12.007+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:12.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:12.007+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:12.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:12 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:12.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:12.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:12 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:12.250+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:12.959 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:12 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:12.639+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:13.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:13 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:13.024+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:13.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:13 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:13.024+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:13.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:12.940+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:13.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:13.202+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:13.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:12 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:12.960+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:13.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:12 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:12.960+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:13.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:13 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:13.163+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:13.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:13 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:13.163+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:13.954 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:13.630+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:13.954 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:13.630+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:14.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:14 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:14.053+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:14.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:14 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:14.053+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:14.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:13 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:13.941+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:14.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:13 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:13.941+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:14.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:14.171+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:14.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:13.955+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:14.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:13 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:13.955+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:14.331 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:14 smithi150 podman[74216]: 2024-05-14 17:20:14.028825219 +0000 UTC m=+5.564275992 container died 6034dcd46af2ec9ce951d7ec316f1b8631471fdbfe079547045ba9919e53434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-14T17:20:14.331 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:14 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:14.185+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:14.331 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:14 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:14.185+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:14.587 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:14 smithi150 podman[74216]: 2024-05-14 17:20:14.422386618 +0000 UTC m=+5.957837389 container cleanup 6034dcd46af2ec9ce951d7ec316f1b8631471fdbfe079547045ba9919e53434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-14T17:20:14.587 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:14 smithi150 bash[74216]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4 2024-05-14T17:20:14.952 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:14 smithi150 podman[74241]: 2024-05-14 17:20:14.648955439 +0000 UTC m=+0.617619976 container remove 6034dcd46af2ec9ce951d7ec316f1b8631471fdbfe079547045ba9919e53434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0) 2024-05-14T17:20:14.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:14.608+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:14.977 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:14.608+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:15.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:14 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:14.955+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:15.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:14 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:14.955+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:15.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:15 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:15.057+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:15.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:15 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:15.057+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:15.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:15.125+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:15.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:14.978+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:15.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:14 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:14.978+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:15.346 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:15.005+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:15.346 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:15.229+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:15.346 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:15.229+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:15.347 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:15 smithi150 podman[74331]: 2024-05-14 17:20:14.988926979 +0000 UTC m=+0.016990464 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:15.347 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:15 smithi150 podman[74331]: 2024-05-14 17:20:15.108280354 +0000 UTC m=+0.136343870 container create cf8ad64d5f8f876b7967bd5615860659b94e1d573f1d8bd59ed6a367bee4c0ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:15.646 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:15 smithi150 podman[74331]: 2024-05-14 17:20:15.34731074 +0000 UTC m=+0.375374215 container init cf8ad64d5f8f876b7967bd5615860659b94e1d573f1d8bd59ed6a367bee4c0ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:20:15.647 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:15 smithi150 podman[74331]: 2024-05-14 17:20:15.350768441 +0000 UTC m=+0.378831927 container start cf8ad64d5f8f876b7967bd5615860659b94e1d573f1d8bd59ed6a367bee4c0ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:15.647 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:15 smithi150 podman[74331]: 2024-05-14 17:20:15.444226029 +0000 UTC m=+0.472289513 container attach cf8ad64d5f8f876b7967bd5615860659b94e1d573f1d8bd59ed6a367bee4c0ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T17:20:15.647 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:15 smithi150 podman[74331]: 2024-05-14 17:20:15.648058252 +0000 UTC m=+0.676121727 container died cf8ad64d5f8f876b7967bd5615860659b94e1d573f1d8bd59ed6a367bee4c0ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:15.970 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:15.586+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:15.970 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:15.586+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:16.163 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:15 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:15.978+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:16.163 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:16 smithi150 podman[74357]: 2024-05-14 17:20:16.138982504 +0000 UTC m=+0.481798390 container remove cf8ad64d5f8f876b7967bd5615860659b94e1d573f1d8bd59ed6a367bee4c0ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True) 2024-05-14T17:20:16.163 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:16 smithi150 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-14T17:20:16.163 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:16 smithi150 systemd[1]: Stopped Ceph osd.4 for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:20:16.164 INFO:journalctl@ceph.osd.4.smithi150.stdout:May 14 17:20:16 smithi150 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.4.service: Consumed 42.663s CPU time. 2024-05-14T17:20:16.165 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.4.service' 2024-05-14T17:20:16.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:16.204 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-14T17:20:16.205 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-14T17:20:16.205 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.5 2024-05-14T17:20:16.231 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:16.132+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:16.232 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:15.971+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:16.232 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:15 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:15.971+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:16.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:16.046+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:16.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:16.046+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:16.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:15 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:15.917+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:16.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:15 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:15.917+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:16.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:16 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:16.197+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:16.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:16 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:16.197+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:16.524 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:16 smithi193 systemd[1]: Stopping Ceph osd.5 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:16.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:16.596+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:16.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:16.596+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:16.792 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:16.525+0000 7fa1992b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:16.793 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:16.525+0000 7fa1992b4700 -1 osd.5 108 *** Got signal Terminated *** 2024-05-14T17:20:16.793 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:16.525+0000 7fa1992b4700 -1 osd.5 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:17.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:17 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:17.058+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:17.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:17 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:17.058+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:17.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:16.893+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:17.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:16.893+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:17.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:16.975+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:17.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:16 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:16.975+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:17.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:17.119+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:17.452 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:16 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:16.979+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:17.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:17.175+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:17.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:17.175+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:17.956 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:17.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:17.957 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:17.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:18.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:18.073+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:18.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:18.073+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:18.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:17 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:17.939+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:18.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:17 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:17.939+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:18.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:17.958+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:18.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:17 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:17.958+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:18.293 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:18.167+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:18.453 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:17 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:17.989+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:18.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:18.222+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:18.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:18.222+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:18.974 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:18.583+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:18.974 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:18.583+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:19.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:19.059+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:19.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:19.059+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:19.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:18.944+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:19.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:18 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:18.944+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:19.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:19.152+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:19.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:18.974+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:19.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:18.974+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:19.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:18 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:18.974+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.777547+0000 front 2024-05-14T17:19:57.777610+0000 (oldest deadline 2024-05-14T17:20:18.877674+0000) 2024-05-14T17:20:19.453 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:18 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:18.970+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:19.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:19 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:19.189+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:19.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:19 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:19.189+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:19.962 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:19.613+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:19.962 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:19.613+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:19.963 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:19.613+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:20.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:19 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:19.951+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:20.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:20.036+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:20.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:20.036+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:20.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:20.036+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:20.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:19.960+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:20.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:19.960+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:20.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:19 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:19.960+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:20.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:20.162+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:20.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:20.162+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:20.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:19.963+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:20.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:19.963+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:20.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:19 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:19.963+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.777547+0000 front 2024-05-14T17:19:57.777610+0000 (oldest deadline 2024-05-14T17:20:18.877674+0000) 2024-05-14T17:20:20.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:20.216+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:20.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:20.216+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:20.914 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:20.626+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:20.915 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:20.626+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:20.915 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:20.626+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:21.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:20 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:20.945+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:21.203 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:21.171+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:21.203 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:21.171+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:21.203 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:21.171+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:21.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:21.060+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:21.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:21.060+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:21.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:21.060+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:21.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:20.962+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:21.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:20.962+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:21.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:20 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:20.962+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:21.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:21 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:21.136+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:21.293 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:21 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:21.136+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:21.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:20.916+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.573916+0000 front 2024-05-14T17:19:38.573677+0000 (oldest deadline 2024-05-14T17:20:04.473896+0000) 2024-05-14T17:20:21.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:20.916+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.875833+0000 front 2024-05-14T17:19:47.875887+0000 (oldest deadline 2024-05-14T17:20:11.375918+0000) 2024-05-14T17:20:21.294 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:20 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5[47981]: 2024-05-14T17:20:20.916+0000 7fa1918a0700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.777547+0000 front 2024-05-14T17:19:57.777610+0000 (oldest deadline 2024-05-14T17:20:18.877674+0000) 2024-05-14T17:20:22.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:21 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:21.613+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:22.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:21 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:21.613+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:22.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:21 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:21.613+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:22.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:21 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:21.920+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:22.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:22.041+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:22.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:22.041+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:22.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:22.041+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:22.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:21.966+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:22.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:21.966+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:22.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:21 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:21.966+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:22.392 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:22 smithi193 podman[75985]: 2024-05-14 17:20:22.089660306 +0000 UTC m=+5.678170679 container died 351fd60d2be88fff19da2daa806e05755c34fe93dec7b12066c44229bd055aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:22.392 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:22 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:22.108+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:22.392 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:22 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:22.108+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:22.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:22 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:22.205+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:22.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:22 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:22.205+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:22.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:22 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:22.205+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:22.716 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:22 smithi193 podman[75985]: 2024-05-14 17:20:22.483613602 +0000 UTC m=+6.072123968 container cleanup 351fd60d2be88fff19da2daa806e05755c34fe93dec7b12066c44229bd055aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True) 2024-05-14T17:20:22.716 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:22 smithi193 bash[75985]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5 2024-05-14T17:20:22.717 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:22 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:22.622+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:22.717 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:22 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:22.622+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:22.717 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:22 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:22.622+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:23.034 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:22 smithi193 podman[76001]: 2024-05-14 17:20:22.717950202 +0000 UTC m=+0.623817251 container remove 351fd60d2be88fff19da2daa806e05755c34fe93dec7b12066c44229bd055aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-14T17:20:23.170 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:22 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:22.888+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:23.170 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:22 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:22.888+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:23.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:23 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:23.035+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:23.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:23 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:23.035+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:23.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:23 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:23.035+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:23.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:22.982+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:23.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:22.982+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:23.265 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:22.982+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:23.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:23 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:23.093+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:23.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:23 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:23.093+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:23.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:23 smithi193 podman[76092]: 2024-05-14 17:20:22.938017933 +0000 UTC m=+0.017606034 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:23.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:23 smithi193 podman[76092]: 2024-05-14 17:20:23.052491983 +0000 UTC m=+0.132080069 container create 0750058692dbdb32f297fea7b8353e1bb4b4bef64d055753036d1c4b7fe4a85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:23.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:23 smithi193 podman[76092]: 2024-05-14 17:20:23.263984879 +0000 UTC m=+0.343572968 container init 0750058692dbdb32f297fea7b8353e1bb4b4bef64d055753036d1c4b7fe4a85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-14T17:20:23.293 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:23 smithi193 podman[76092]: 2024-05-14 17:20:23.266962822 +0000 UTC m=+0.346550911 container start 0750058692dbdb32f297fea7b8353e1bb4b4bef64d055753036d1c4b7fe4a85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:23.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:23.171+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:23.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:23.171+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:23.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:23.171+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:23.792 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:23 smithi193 podman[76092]: 2024-05-14 17:20:23.346595507 +0000 UTC m=+0.426183604 container attach 0750058692dbdb32f297fea7b8353e1bb4b4bef64d055753036d1c4b7fe4a85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-14T17:20:23.793 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:23 smithi193 podman[76092]: 2024-05-14 17:20:23.557753454 +0000 UTC m=+0.637341552 container died 0750058692dbdb32f297fea7b8353e1bb4b4bef64d055753036d1c4b7fe4a85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-14T17:20:23.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:23 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:23.592+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:23.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:23 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:23.592+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:23.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:23 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:23.592+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:24.103 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 17:20:24 smithi193 podman[76118]: 2024-05-14 17:20:24.083346752 +0000 UTC m=+0.519935964 container remove 0750058692dbdb32f297fea7b8353e1bb4b4bef64d055753036d1c4b7fe4a85b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-14T17:20:24.108 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.5.service' 2024-05-14T17:20:24.148 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:23.855+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:24.148 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:23 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:23.855+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:24.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:24.186 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-14T17:20:24.187 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-14T17:20:24.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.6 2024-05-14T17:20:24.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:24 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:24.148+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:24.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:24 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:24.148+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:24.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:24 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:24.148+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:24.482 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:24.031+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:24.482 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:24.031+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:24.482 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:24.031+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:24.482 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:24.055+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:24.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:24.055+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:24.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:24.055+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:24.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 systemd[1]: Stopping Ceph osd.6 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:24.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:24.482+0000 7f0b0bd42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:24.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:24.482+0000 7f0b0bd42700 -1 osd.6 108 *** Got signal Terminated *** 2024-05-14T17:20:24.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:24 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:24.482+0000 7f0b0bd42700 -1 osd.6 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:24.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:24 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:24.130+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:24.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:24 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:24.130+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:24.953 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:24 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:24.872+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:24.953 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:24 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:24.872+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:25.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:24 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:24.608+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:25.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:24 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:24.608+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:25.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:24 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:24.608+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:25.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:25.148+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:25.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:25.148+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:25.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:25.148+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:25.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:25 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:25.017+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:25.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:25 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:25.017+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:25.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:25 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:25.017+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:25.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:25 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:25.058+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:25.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:25 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:25.058+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:25.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:25 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:25.058+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:25.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:25 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:25.143+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:25.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:25 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:25.143+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:26.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:25 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:25.649+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:26.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:25 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:25.649+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:26.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:25 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:25.649+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:26.054 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:26 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:26.031+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:26.054 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:26 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:26.031+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:26.054 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:26 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:26.031+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:26.128 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:25.878+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:26.128 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:25.878+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:26.128 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:26 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:26.128+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:26.129 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:26 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:26.128+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:26.129 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:26 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:26.128+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:26.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:26 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:26.055+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:26.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:26 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:26.055+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:26.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:26 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:26.055+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:26.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:26 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:26.132+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:26.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:26 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:26.132+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:27.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:26 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:26.639+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:27.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:26 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:26.639+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:27.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:26 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:26.639+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:27.171 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:26 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:26.921+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:27.172 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:26 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:26.921+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:27.452 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:27 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:27.172+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:27.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:27 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:27.172+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:27.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:27 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:27.172+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:27.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:27 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:27.018+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:27.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:27 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:27.018+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:27.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:27 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:27.018+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:27.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:27 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:27.067+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:27.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:27 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:27.067+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:27.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:27 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:27.067+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:27.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:27 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:27.138+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:27.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:27 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:27.138+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:28.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:27 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:27.605+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:28.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:27 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:27.605+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:28.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:27 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:27.605+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:28.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:27 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:27.605+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:28.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:27 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:27.924+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:28.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:27 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:27.924+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:28.514 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:28.103+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:28.514 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:28.103+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:28.514 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:28.103+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:28.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:28.036+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:28.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:28.036+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:28.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:28 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:28.036+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:28.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:28.089+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:28.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:28.089+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:28.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:28.089+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:28.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:28.203+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:28.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:28.203+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:28.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:28.203+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:29.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:28.615+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:29.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:28.615+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:29.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:28.615+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:29.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:28 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:28.615+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:29.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:28.937+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:29.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:28.937+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:29.514 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:29.106+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.725527+0000 front 2024-05-14T17:19:39.725502+0000 (oldest deadline 2024-05-14T17:20:03.225719+0000) 2024-05-14T17:20:29.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:29.106+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:46.126756+0000 front 2024-05-14T17:19:46.126669+0000 (oldest deadline 2024-05-14T17:20:10.826700+0000) 2024-05-14T17:20:29.515 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6[57905]: 2024-05-14T17:20:29.106+0000 7f0b0432e700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.228220+0000 front 2024-05-14T17:19:57.228300+0000 (oldest deadline 2024-05-14T17:20:19.528113+0000) 2024-05-14T17:20:29.516 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:29.073+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:29.516 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:29.073+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:29.516 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:29.073+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:29.516 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:29 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:29.073+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:29.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:29.076+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:29.545 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:29.076+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:29.545 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:29.076+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:29.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:29 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:29.226+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:29.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:29 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:29.226+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:29.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:29 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:29.226+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:30.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:29.617+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:30.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:29.617+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:30.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:29.617+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:30.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:29 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:29.617+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:30.115 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:29 smithi099 podman[106860]: 2024-05-14 17:20:29.844351908 +0000 UTC m=+5.455710869 container died b2c2d9927caa1b9d5cb8d4115d85e73ad8752ec3755d90c2f98ed9a9699ae2ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-14T17:20:30.116 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:30.084+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:30.116 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:30.084+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:30.116 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:30.084+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:30.116 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:30 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:30.084+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:30.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:29 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:29.921+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:30.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:29 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:29.921+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:30.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:29 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:29.921+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:30.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:30 smithi099 podman[106860]: 2024-05-14 17:20:30.282451859 +0000 UTC m=+5.893810849 container cleanup b2c2d9927caa1b9d5cb8d4115d85e73ad8752ec3755d90c2f98ed9a9699ae2ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:30.483 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:30 smithi099 bash[106860]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6 2024-05-14T17:20:30.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:30.084+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:30.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:30.084+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:30.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:30.084+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:30.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:30.260+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:30.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:30.260+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:30.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:30.260+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:30.764 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:30 smithi099 podman[106887]: 2024-05-14 17:20:30.484104775 +0000 UTC m=+0.638489200 container remove b2c2d9927caa1b9d5cb8d4115d85e73ad8752ec3755d90c2f98ed9a9699ae2ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:31.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:30.631+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:31.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:30.631+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:31.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:30.631+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:31.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:30 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:30.631+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:31.203 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:30 smithi099 podman[106979]: 2024-05-14 17:20:30.802819711 +0000 UTC m=+0.017328374 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:31.203 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:30 smithi099 podman[106979]: 2024-05-14 17:20:30.909968755 +0000 UTC m=+0.124477463 container create 2fd93a78e8ae8c8db1aaf68cd8b8bd01acf4ce85f6baab8dc0e39cc43a308dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-14T17:20:31.203 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:31 smithi099 podman[106979]: 2024-05-14 17:20:31.128931335 +0000 UTC m=+0.343439999 container init 2fd93a78e8ae8c8db1aaf68cd8b8bd01acf4ce85f6baab8dc0e39cc43a308dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-14T17:20:31.203 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:31 smithi099 podman[106979]: 2024-05-14 17:20:31.132281075 +0000 UTC m=+0.346789744 container start 2fd93a78e8ae8c8db1aaf68cd8b8bd01acf4ce85f6baab8dc0e39cc43a308dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-14T17:20:31.203 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:31 smithi099 podman[106979]: 2024-05-14 17:20:31.203656844 +0000 UTC m=+0.418165522 container attach 2fd93a78e8ae8c8db1aaf68cd8b8bd01acf4ce85f6baab8dc0e39cc43a308dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:31.204 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:31.093+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:31.204 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:31.093+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:31.204 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:31.093+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:31.204 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:31 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:31.093+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:31.204 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:30.873+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:31.205 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:30.873+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:31.205 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:30 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:30.873+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:31.514 INFO:journalctl@ceph.osd.6.smithi099.stdout:May 14 17:20:31 smithi099 podman[106979]: 2024-05-14 17:20:31.436952583 +0000 UTC m=+0.651461246 container died 2fd93a78e8ae8c8db1aaf68cd8b8bd01acf4ce85f6baab8dc0e39cc43a308dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:31.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:31.092+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:31.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:31.092+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:31.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:31.092+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:31.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:31.281+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:31.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:31.281+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:31.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:31.281+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:31.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:31.281+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:32.027 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.6.service' 2024-05-14T17:20:32.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:31.597+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:32.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:31.597+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:32.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:31.597+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:32.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:31 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:31.597+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:32.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:32.067 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-14T17:20:32.067 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-14T17:20:32.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.7 2024-05-14T17:20:32.115 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:31.860+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:32.115 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:31.860+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:32.115 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:31.860+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:32.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:32.111+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:32.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:32.111+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:32.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:32.111+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:32.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:32 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:32.111+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:32.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 systemd[1]: Stopping Ceph osd.7 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:32.382 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:32.297+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:32.382 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:32.297+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:32.382 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:32.297+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:32.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:32.297+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:32.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:32.101+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:32.543 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:32.101+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:32.543 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:32.101+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:32.703 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:32.383+0000 7f81c4bb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:32.703 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:32.383+0000 7f81c4bb5700 -1 osd.7 108 *** Got signal Terminated *** 2024-05-14T17:20:32.703 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:32.383+0000 7f81c4bb5700 -1 osd.7 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:33.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:32.555+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:33.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:32.555+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:33.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:32.555+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:33.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:32 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:32.555+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:33.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:32.849+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:33.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:32.849+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:33.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:32 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:32.849+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:33.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:33.130+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:33.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:33.130+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:33.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:33.130+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:33.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:33 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:33.130+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:33.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:33.119+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:33.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:33.119+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:33.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:33.119+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:33.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:33.275+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:33.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:33.275+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:33.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:33.275+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:33.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:33.275+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:34.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:33.570+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:34.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:33.570+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:34.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:33.570+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:34.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:33 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:33.570+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:34.202 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:33.886+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:34.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:33.886+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:34.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:33 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:33.886+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:34.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:34.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:34.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:34.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:34.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:34.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:34.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:34 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:34.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:34.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:34.153+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:34.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:34.153+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:34.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:34.153+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:34.702 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:34.235+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:34.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:34.235+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:34.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:34.235+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:34.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:34.235+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:35.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:34.602+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:35.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:34.602+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:35.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:34.602+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:35.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:34 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:34.602+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:35.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:34.870+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:35.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:34.870+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:35.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:34.870+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:35.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:34 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:34.870+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.301951+0000 front 2024-05-14T17:20:10.301924+0000 (oldest deadline 2024-05-14T17:20:34.402044+0000) 2024-05-14T17:20:35.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:35.087+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:35.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:35.087+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:35.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:35.087+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:35.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:35 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:35.087+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:35.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:35.185+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:35.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:35.185+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:35.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:35.185+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:35.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:35.267+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:35.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:35.267+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:35.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:35.267+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:35.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:35.267+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:35.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:35.267+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:36.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:35.603+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:36.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:35.603+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:36.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:35.603+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:36.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:35 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:35.603+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:36.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:35.875+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:36.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:35.875+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:36.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:35.875+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:36.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:35 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:35.875+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.301951+0000 front 2024-05-14T17:20:10.301924+0000 (oldest deadline 2024-05-14T17:20:34.402044+0000) 2024-05-14T17:20:36.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:36.092+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:36.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:36.092+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:36.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:36.092+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:36.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:36 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:36.092+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:36.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:36.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:36.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:36.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:36.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:36.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:36.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:36.217+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:36.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:36.217+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:36.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:36.217+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:36.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:36.217+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:36.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:36.217+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:37.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:36.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:37.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:36.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:37.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:36.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:37.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:36.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:37.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:36 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:36.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:37.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:36.871+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:49.397678+0000 front 2024-05-14T17:19:49.397702+0000 (oldest deadline 2024-05-14T17:20:14.097888+0000) 2024-05-14T17:20:37.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:36.871+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:55.799264+0000 front 2024-05-14T17:19:55.799099+0000 (oldest deadline 2024-05-14T17:20:22.199705+0000) 2024-05-14T17:20:37.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:36.871+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.100978+0000 front 2024-05-14T17:20:05.101041+0000 (oldest deadline 2024-05-14T17:20:29.201020+0000) 2024-05-14T17:20:37.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7[54086]: 2024-05-14T17:20:36.871+0000 7f81c11a9700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.301951+0000 front 2024-05-14T17:20:10.301924+0000 (oldest deadline 2024-05-14T17:20:34.402044+0000) 2024-05-14T17:20:37.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:37.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:37.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:37.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:37.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:37.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:37.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:37 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:37.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:37.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:37.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:37.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:37.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:37.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:37.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:37.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:37.256+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:37.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:37.256+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:37.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:37.256+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:37.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:37.256+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:37.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:37 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:37.256+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:38.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:37.546+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:38.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:37.546+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:38.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:37.546+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:38.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:37.546+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:38.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:37 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:37.546+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:38.048 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:37 smithi150 podman[74439]: 2024-05-14 17:20:37.745442901 +0000 UTC m=+5.480894285 container died b6fc71cacc1ca8d5e62100c1ce05ee19d7e1116aee7e294cf7a700282ec41c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:20:38.048 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74439]: 2024-05-14 17:20:38.049848566 +0000 UTC m=+5.785299949 container cleanup b6fc71cacc1ca8d5e62100c1ce05ee19d7e1116aee7e294cf7a700282ec41c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-14T17:20:38.453 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 bash[74439]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7 2024-05-14T17:20:38.453 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74455]: 2024-05-14 17:20:38.267543294 +0000 UTC m=+0.520333144 container remove b6fc71cacc1ca8d5e62100c1ce05ee19d7e1116aee7e294cf7a700282ec41c94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T17:20:38.454 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:38 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:38.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:38.454 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:38 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:38.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:38.454 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:38 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:38.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:38.455 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:38 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:38.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:38.455 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:38 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:38.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:38.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:38 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:38.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:38.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:38 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:38.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:38.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:38 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:38.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:38.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:38 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:38.113+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:38.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:38.182+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:38.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:38.182+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:38.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:38.182+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:38.543 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:38.182+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:38.896 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74544]: 2024-05-14 17:20:38.593247699 +0000 UTC m=+0.114054485 container create 41d55c74fd9ca20d27e01150f3549b190c6df13301f800dff7754012468a1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:38.897 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74544]: 2024-05-14 17:20:38.49651422 +0000 UTC m=+0.017321024 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:38.897 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74544]: 2024-05-14 17:20:38.804532127 +0000 UTC m=+0.325338933 container init 41d55c74fd9ca20d27e01150f3549b190c6df13301f800dff7754012468a1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:20:38.897 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74544]: 2024-05-14 17:20:38.807786498 +0000 UTC m=+0.328593295 container start 41d55c74fd9ca20d27e01150f3549b190c6df13301f800dff7754012468a1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:38.897 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:38 smithi150 podman[74544]: 2024-05-14 17:20:38.897823728 +0000 UTC m=+0.418630525 container attach 41d55c74fd9ca20d27e01150f3549b190c6df13301f800dff7754012468a1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:39.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:38.565+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:39.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:38.565+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:39.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:38.565+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:39.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:38.565+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:39.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:38 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:38.565+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:39.203 INFO:journalctl@ceph.osd.7.smithi150.stdout:May 14 17:20:39 smithi150 podman[74544]: 2024-05-14 17:20:39.104104686 +0000 UTC m=+0.624911491 container died 41d55c74fd9ca20d27e01150f3549b190c6df13301f800dff7754012468a1350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T17:20:39.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:39 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:39.081+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:39.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:39 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:39.081+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:39.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:39 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:39.081+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:39.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:39 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:39.081+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:39.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:39.193+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:39.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:39.193+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:39.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:39.193+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:39.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:39.193+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:39.631 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:39 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:39.309+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:39.632 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:39 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:39.309+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:39.632 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:39 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:39.309+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:39.632 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:39 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:39.309+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:39.632 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:39 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:39.309+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:39.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.7.service' 2024-05-14T17:20:39.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:39.686 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-14T17:20:39.686 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-14T17:20:39.687 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.8 2024-05-14T17:20:40.010 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:39 smithi193 systemd[1]: Stopping Ceph osd.8 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:40.010 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:39.553+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:40.010 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:39.553+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:40.011 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:39.553+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:40.011 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:39.553+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:40.011 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:39 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:39.553+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:40.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.010+0000 7f860df7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:40.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.010+0000 7f860df7e700 -1 osd.8 108 *** Got signal Terminated *** 2024-05-14T17:20:40.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.010+0000 7f860df7e700 -1 osd.8 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:40.292 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.227+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:40.293 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.227+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:40.293 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.227+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:40.293 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:40.227+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:40.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:40 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:40.127+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:40.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:40 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:40.127+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:40.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:40 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:40.127+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:40.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:40 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:40.127+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:40.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:40 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:40.341+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:40.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:40 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:40.341+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:40.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:40 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:40.341+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:40.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:40 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:40.341+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:40.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:40 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:40.341+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:41.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:40.593+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:41.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:40.593+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:41.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:40.593+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:41.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:40.593+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:41.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:40.593+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:41.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:41.177+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:41.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:41.177+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:41.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:41.177+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:41.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:41 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:41.177+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:41.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:41.181+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:41.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:41.181+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:41.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:41.181+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:41.542 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:41.181+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:41.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:41 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:41.326+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:41.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:41 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:41.326+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:41.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:41 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:41.326+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:41.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:41 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:41.326+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:41.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:41 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:41.326+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:42.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:41.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:42.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:41.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:42.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:41.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:42.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:41.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:42.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:41 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:41.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:42.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:42 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:42.209+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:42 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:42.209+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:42 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:42.209+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:42 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:42.209+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:42.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:42.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:42.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:42.515 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:42.145+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:42.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:42 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:42.298+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:42.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:42 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:42.298+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:42.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:42 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:42.298+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:42.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:42 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:42.298+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:42.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:42 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:42.298+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:42.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:42.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:42.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:42.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:42.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:42.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:42.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:42.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:42.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:42 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:42.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:43.509 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:43.170+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:43.509 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:43.170+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:43.510 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:43.170+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:43.510 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:43.170+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:43.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:43 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:43.215+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:43.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:43 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:43.215+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:43.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:43 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:43.215+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:43.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:43 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:43.215+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:43.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:43 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:43.215+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:43.604 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:43 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:43.335+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:43.604 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:43 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:43.335+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:43.604 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:43 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:43.335+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:43.605 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:43 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:43.335+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:43.605 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:43 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:43.335+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:43.605 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:43 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:43.335+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:43.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:43.510+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:43.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:43.510+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:43.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:43.510+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:43.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:43.510+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:43.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:43 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:43.510+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:44.472 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:44.206+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.066035+0000 front 2024-05-14T17:19:48.066049+0000 (oldest deadline 2024-05-14T17:20:10.965990+0000) 2024-05-14T17:20:44.472 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:44.206+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.768078+0000 front 2024-05-14T17:19:56.768106+0000 (oldest deadline 2024-05-14T17:20:19.668212+0000) 2024-05-14T17:20:44.472 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:44.206+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:01.869628+0000 front 2024-05-14T17:20:01.869712+0000 (oldest deadline 2024-05-14T17:20:27.169747+0000) 2024-05-14T17:20:44.473 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8[54941]: 2024-05-14T17:20:44.206+0000 7f860656a700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.471065+0000 front 2024-05-14T17:20:12.471089+0000 (oldest deadline 2024-05-14T17:20:37.671968+0000) 2024-05-14T17:20:44.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:44 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:44.186+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:44.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:44 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:44.186+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:44.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:44 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:44.186+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:44.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:44 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:44.186+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:44.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:44 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:44.186+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:44.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:44.381+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:44.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:44.381+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:44.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:44.381+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:44.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:44.381+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:44.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:44.381+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:44.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:44 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:44.381+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:44.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:44.473+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:44.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:44.473+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:44.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:44.473+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:44.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:44.473+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:44.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:44.473+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:44.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:44.473+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:45.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:45 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:45.208+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:45.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:45 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:45.208+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:45.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:45 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:45.208+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:45.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:45 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:45.208+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:45.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:45 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:45.208+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:45.667 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:45 smithi193 podman[76198]: 2024-05-14 17:20:45.353298539 +0000 UTC m=+5.475263050 container died 88e88f839ec0a7cdc244e6431b6db311c7dc31f8bbd8fe0d2115cd96ff5ba6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T17:20:45.667 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:45 smithi193 podman[76198]: 2024-05-14 17:20:45.668572306 +0000 UTC m=+5.790536810 container cleanup 88e88f839ec0a7cdc244e6431b6db311c7dc31f8bbd8fe0d2115cd96ff5ba6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:45.667 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:45 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:45.517+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:45.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:45 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:45.517+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:45.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:45 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:45.517+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:45.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:45 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:45.517+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:45.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:45 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:45.517+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:45.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:45 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:45.517+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:45.706 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:45 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:45.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:45.707 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:45 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:45.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:45.707 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:45 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:45.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:45.707 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:45 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:45.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:45.707 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:45 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:45.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:45.707 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:45 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:45.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:46.042 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:45 smithi193 bash[76198]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8 2024-05-14T17:20:46.042 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:45 smithi193 podman[76216]: 2024-05-14 17:20:45.869927526 +0000 UTC m=+0.514640829 container remove 88e88f839ec0a7cdc244e6431b6db311c7dc31f8bbd8fe0d2115cd96ff5ba6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:20:46.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:46 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:46.249+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:46.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:46 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:46.249+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:46.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:46 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:46.249+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:46.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:46 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:46.249+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:46.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:46 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:46.249+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:46.530 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:46 smithi193 podman[76303]: 2024-05-14 17:20:46.262333759 +0000 UTC m=+0.092148582 container create ee56fc08373d880589ba98400bfc7133755f8208cf8eb7e87625819f0f256911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:20:46.530 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:46 smithi193 podman[76303]: 2024-05-14 17:20:46.187529154 +0000 UTC m=+0.017343969 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:46.530 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:46 smithi193 podman[76303]: 2024-05-14 17:20:46.448471742 +0000 UTC m=+0.278286543 container init ee56fc08373d880589ba98400bfc7133755f8208cf8eb7e87625819f0f256911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-14T17:20:46.530 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:46 smithi193 podman[76303]: 2024-05-14 17:20:46.45144306 +0000 UTC m=+0.281257862 container start ee56fc08373d880589ba98400bfc7133755f8208cf8eb7e87625819f0f256911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2) 2024-05-14T17:20:46.531 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:46 smithi193 podman[76303]: 2024-05-14 17:20:46.531795473 +0000 UTC m=+0.361610274 container attach ee56fc08373d880589ba98400bfc7133755f8208cf8eb7e87625819f0f256911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-14T17:20:46.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:46 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:46.336+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:46.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:46 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:46.336+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:46.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:46 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:46.336+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:46.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:46 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:46.336+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:46.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:46 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:46.336+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:46.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:46 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:46.336+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:46.792 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 17:20:46 smithi193 podman[76303]: 2024-05-14 17:20:46.747660604 +0000 UTC m=+0.577475412 container died ee56fc08373d880589ba98400bfc7133755f8208cf8eb7e87625819f0f256911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-05-14T17:20:46.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:46 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:46.550+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:46.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:46 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:46.550+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:46.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:46 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:46.550+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:46.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:46 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:46.550+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:46.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:46 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:46.550+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:46.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:46 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:46.550+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:47.328 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.8.service' 2024-05-14T17:20:47.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:47.368 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-14T17:20:47.369 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-14T17:20:47.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.9 2024-05-14T17:20:47.666 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.274+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:47.666 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.274+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.274+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.274+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.274+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 systemd[1]: Stopping Ceph osd.9 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.666+0000 7f95c1c8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.666+0000 7f95c1c8a700 -1 osd.9 108 *** Got signal Terminated *** 2024-05-14T17:20:47.667 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:47 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:47.666+0000 7f95c1c8a700 -1 osd.9 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:47.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:47 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:47.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:47.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:47 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:47.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:47.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:47 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:47.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:47.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:47 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:47.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:47.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:47 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:47.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:47.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:47 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:47.287+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:48.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:47 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:47.568+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:48.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:47 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:47.568+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:48.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:47 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:47.568+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:48.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:47 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:47.568+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:48.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:47 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:47.568+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:48.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:47 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:47.568+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:48.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:48 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:48.265+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:48.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:48 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:48.265+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:48.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:48 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:48.265+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:48.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:48 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:48.265+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:48.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:48 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:48.265+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:48.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:48 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:48.265+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:48.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:48 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:48.271+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:48.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:48 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:48.271+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:48.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:48 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:48.271+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:48.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:48 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:48.271+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:48.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:48 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:48.271+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:48.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:48.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:48.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:48.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:48.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:48.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:48.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:48.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:48.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:48.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:48.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:48.612+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:49.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:49 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:49.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:49.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:49 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:49.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:49.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:49 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:49.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:49.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:49 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:49.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:49.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:49 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:49.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:49.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:49 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:49.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:49.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:49 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:49.318+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:49.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:49 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:49.318+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:49.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:49 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:49.318+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:49.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:49 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:49.318+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:49.765 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:49 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:49.318+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:50.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:49 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:49.648+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:50.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:49 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:49.648+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:50.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:49 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:49.648+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:50.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:49 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:49.648+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:50.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:49 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:49.648+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:50.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:49 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:49.648+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:50.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:50 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:50.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:50.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:50 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:50.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:50.453 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:50 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:50.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:50.454 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:50 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:50.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:50.454 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:50 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:50.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:50.454 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:50 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:50.177+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:50.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:50 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:50.330+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:50.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:50 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:50.330+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:50.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:50 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:50.330+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:50.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:50 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:50.330+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:50.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:50 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:50.330+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:51.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:50 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:50.667+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:51.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:50 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:50.667+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:51.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:50 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:50.667+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:51.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:50 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:50.667+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:51.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:50 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:50.667+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:51.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:50 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:50.667+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:51.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:51 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:51.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:51.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:51 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:51.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:51.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:51 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:51.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:51.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:51 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:51.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:51.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:51 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:51.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:51.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:51 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:51.227+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:51.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:51 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:51.295+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:51.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:51 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:51.295+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:51.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:51 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:51.295+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:51.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:51 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:51.295+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:51.765 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:51 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:51.295+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:52.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:51 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:51.629+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:52.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:51 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:51.629+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:52.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:51 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:51.629+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:52.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:51 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:51.629+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:52.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:51 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:51.629+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:52.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:51 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:51.629+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:52.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:52 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:52.260+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:39.145056+0000 front 2024-05-14T17:19:39.145147+0000 (oldest deadline 2024-05-14T17:20:03.245154+0000) 2024-05-14T17:20:52.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:52 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:52.260+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:47.846887+0000 front 2024-05-14T17:19:47.846940+0000 (oldest deadline 2024-05-14T17:20:13.146837+0000) 2024-05-14T17:20:52.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:52 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:52.260+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:56.047757+0000 front 2024-05-14T17:19:56.047817+0000 (oldest deadline 2024-05-14T17:20:19.547728+0000) 2024-05-14T17:20:52.514 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:52 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:52.260+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.448700+0000 front 2024-05-14T17:20:05.448769+0000 (oldest deadline 2024-05-14T17:20:28.348900+0000) 2024-05-14T17:20:52.515 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:52 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9[64780]: 2024-05-14T17:20:52.260+0000 7f95be27e700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:16.451042+0000 front 2024-05-14T17:20:16.450994+0000 (oldest deadline 2024-05-14T17:20:42.351017+0000) 2024-05-14T17:20:52.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:52.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:52.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:52.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:52.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:52.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:52.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:52 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:52.213+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:52.943 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:52.653+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:52.944 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:52.653+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:52.944 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:52.653+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:52.944 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:52.653+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:52.944 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:52.653+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:52.944 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:52.653+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:53.407 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:53 smithi099 podman[107084]: 2024-05-14 17:20:53.115243607 +0000 UTC m=+5.545171226 container died 6570294a430580fec9bd3b15431a417806296a96971b4f4dc6c1a795cf2dbb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-14T17:20:53.660 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:53.661 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:53.661 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:53.661 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:53.661 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:53.661 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:53.661 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:53 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:53.236+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:53.732 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:53 smithi099 podman[107084]: 2024-05-14 17:20:53.574216598 +0000 UTC m=+6.004144171 container cleanup 6570294a430580fec9bd3b15431a417806296a96971b4f4dc6c1a795cf2dbb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-14T17:20:53.732 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:53 smithi099 bash[107084]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9 2024-05-14T17:20:54.014 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:53 smithi099 podman[107100]: 2024-05-14 17:20:53.733840525 +0000 UTC m=+0.612026413 container remove 6570294a430580fec9bd3b15431a417806296a96971b4f4dc6c1a795cf2dbb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:54.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:53 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:53.623+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:54.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:53 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:53.623+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:54.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:53 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:53.623+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:54.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:53 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:53.623+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:54.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:53 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:53.623+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:54.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:53 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:53.623+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:54.427 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:54 smithi099 podman[107188]: 2024-05-14 17:20:54.1262307 +0000 UTC m=+0.100325663 container create fba8f7cbcb15b78fdfccf83f20c62a4362556881722622b0c90808a2c42f4d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T17:20:54.427 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:54 smithi099 podman[107188]: 2024-05-14 17:20:54.042708298 +0000 UTC m=+0.016803257 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:54.428 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:54 smithi099 podman[107188]: 2024-05-14 17:20:54.345379041 +0000 UTC m=+0.319474000 container init fba8f7cbcb15b78fdfccf83f20c62a4362556881722622b0c90808a2c42f4d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-14T17:20:54.428 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:54 smithi099 podman[107188]: 2024-05-14 17:20:54.348528348 +0000 UTC m=+0.322623310 container start fba8f7cbcb15b78fdfccf83f20c62a4362556881722622b0c90808a2c42f4d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:54.428 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:54 smithi099 podman[107188]: 2024-05-14 17:20:54.428686822 +0000 UTC m=+0.402781792 container attach fba8f7cbcb15b78fdfccf83f20c62a4362556881722622b0c90808a2c42f4d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:54.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:54.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:54.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:54.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:54.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:54.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:54.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:54 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:54.239+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:54.764 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:54 smithi099 podman[107188]: 2024-05-14 17:20:54.641772362 +0000 UTC m=+0.615867321 container died fba8f7cbcb15b78fdfccf83f20c62a4362556881722622b0c90808a2c42f4d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-14T17:20:55.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:54 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:54.589+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:55.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:54 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:54.589+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:55.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:54 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:54.589+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:55.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:54 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:54.589+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:55.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:54 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:54.589+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:55.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:54 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:54.589+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:55.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.9.service' 2024-05-14T17:20:55.253 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:55 smithi099 podman[107214]: 2024-05-14 17:20:55.198780062 +0000 UTC m=+0.551030240 container remove fba8f7cbcb15b78fdfccf83f20c62a4362556881722622b0c90808a2c42f4d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:55.253 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:55 smithi099 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.9.service: Deactivated successfully. 2024-05-14T17:20:55.253 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:55 smithi099 systemd[1]: Stopped Ceph osd.9 for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:20:55.253 INFO:journalctl@ceph.osd.9.smithi099.stdout:May 14 17:20:55 smithi099 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.9.service: Consumed 40.518s CPU time. 2024-05-14T17:20:55.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:55.269 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-14T17:20:55.269 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-14T17:20:55.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.10 2024-05-14T17:20:55.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:55.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:55.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:55.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:55.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:55.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:55.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.272+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:55.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 systemd[1]: Stopping Ceph osd.10 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:20:55.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.568+0000 7fabe2cac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:55.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.568+0000 7fabe2cac700 -1 osd.10 108 *** Got signal Terminated *** 2024-05-14T17:20:55.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:55 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:55.568+0000 7fabe2cac700 -1 osd.10 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:56.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:55 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:55.547+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:56.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:55 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:55.547+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:56.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:55 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:55.547+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:56.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:55 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:55.547+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:56.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:55 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:55.547+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:56.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:55 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:55.547+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:56.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:56.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:56.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:56.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:56.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:56.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:56.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:56 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:56.294+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:57.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:56 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:56.567+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:57.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:56 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:56.567+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:57.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:56 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:56.567+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:57.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:56 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:56.567+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:57.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:56 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:56.567+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:57.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:56 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:56.567+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:57.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:57.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:57.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:57.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:57.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:57.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:57.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:57 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:57.301+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:57.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:57 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:57.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:57.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:57 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:57.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:57.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:57 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:57.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:57.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:57 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:57.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:57.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:57 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:57.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:57.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:57 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:57.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:58.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:58 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:58.327+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:59.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:58 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:58.545+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:59.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:58 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:58.545+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:59.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:58 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:58.545+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:59.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:58 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:58.545+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:59.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:58 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:58.545+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:59.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:58 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:58.545+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:20:59.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:20:59.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:20:59.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:20:59.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:20:59.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:20:59.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:20:59.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:20:59 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:20:59.329+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:20:59.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:59 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:59.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:20:59.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:59 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:59.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:20:59.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:59 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:59.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:20:59.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:59 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:59.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:20:59.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:59 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:59.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:20:59.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:20:59 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:20:59.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:00.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:37.751591+0000 front 2024-05-14T17:19:37.751534+0000 (oldest deadline 2024-05-14T17:20:02.451576+0000) 2024-05-14T17:21:00.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.153766+0000 front 2024-05-14T17:19:48.153793+0000 (oldest deadline 2024-05-14T17:20:12.853983+0000) 2024-05-14T17:21:00.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.455780+0000 front 2024-05-14T17:19:57.455911+0000 (oldest deadline 2024-05-14T17:20:20.355701+0000) 2024-05-14T17:21:00.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.374+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.656821+0000 front 2024-05-14T17:20:05.656771+0000 (oldest deadline 2024-05-14T17:20:30.356614+0000) 2024-05-14T17:21:00.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.375+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:10.356979+0000 front 2024-05-14T17:20:10.356925+0000 (oldest deadline 2024-05-14T17:20:35.057052+0000) 2024-05-14T17:21:00.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.375+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.258604+0000 front 2024-05-14T17:20:20.258672+0000 (oldest deadline 2024-05-14T17:20:43.158739+0000) 2024-05-14T17:21:00.704 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:00 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10[60959]: 2024-05-14T17:21:00.375+0000 7fabdf2a0700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-05-14T17:20:29.059984+0000 front 2024-05-14T17:20:29.060081+0000 (oldest deadline 2024-05-14T17:20:51.359686+0000) 2024-05-14T17:21:00.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:00 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:00.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:00.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:00 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:00.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:00.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:00 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:00.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:00.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:00 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:00.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:00.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:00 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:00.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:00.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:00 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:00.518+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:01.383 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:01 smithi150 podman[74660]: 2024-05-14 17:21:01.057707112 +0000 UTC m=+5.587456161 container died 5123cb3fc54c2897704d14f27863092de2e0cf6ab46fd648e2d916c0dab964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-14T17:21:01.384 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:01 smithi150 podman[74660]: 2024-05-14 17:21:01.384874059 +0000 UTC m=+5.914623097 container cleanup 5123cb3fc54c2897704d14f27863092de2e0cf6ab46fd648e2d916c0dab964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T17:21:01.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:01 smithi150 bash[74660]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10 2024-05-14T17:21:01.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:01 smithi150 podman[74677]: 2024-05-14 17:21:01.577693396 +0000 UTC m=+0.521724963 container remove 5123cb3fc54c2897704d14f27863092de2e0cf6ab46fd648e2d916c0dab964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-14T17:21:01.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:01.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:01.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:01.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:01.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:01.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:01.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:01 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:01.504+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:02.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:01 smithi150 podman[74765]: 2024-05-14 17:21:01.902087062 +0000 UTC m=+0.016586547 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:21:02.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:02 smithi150 podman[74765]: 2024-05-14 17:21:02.003509551 +0000 UTC m=+0.118009016 container create c02b280c898e7d4476d9f63920e06f01244491463887818827df6c011a1bfbe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-14T17:21:02.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:02 smithi150 podman[74765]: 2024-05-14 17:21:02.197705257 +0000 UTC m=+0.312204729 container init c02b280c898e7d4476d9f63920e06f01244491463887818827df6c011a1bfbe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:21:02.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:02 smithi150 podman[74765]: 2024-05-14 17:21:02.200697537 +0000 UTC m=+0.315197057 container start c02b280c898e7d4476d9f63920e06f01244491463887818827df6c011a1bfbe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-14T17:21:02.272 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:02 smithi150 podman[74765]: 2024-05-14 17:21:02.272509381 +0000 UTC m=+0.387008863 container attach c02b280c898e7d4476d9f63920e06f01244491463887818827df6c011a1bfbe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-14T17:21:02.703 INFO:journalctl@ceph.osd.10.smithi150.stdout:May 14 17:21:02 smithi150 podman[74765]: 2024-05-14 17:21:02.495129615 +0000 UTC m=+0.609629097 container died c02b280c898e7d4476d9f63920e06f01244491463887818827df6c011a1bfbe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-14T17:21:02.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:02.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:02.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:02.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:02.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:02.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:02.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:02 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:02.533+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:03.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.10.service' 2024-05-14T17:21:03.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:03.053 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-14T17:21:03.053 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-14T17:21:03.053 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.11 2024-05-14T17:21:03.402 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 systemd[1]: Stopping Ceph osd.11 for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:03.403 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.403+0000 7fb1506c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:03.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.403+0000 7fb1506c5700 -1 osd.11 108 *** Got signal Terminated *** 2024-05-14T17:21:03.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.403+0000 7fb1506c5700 -1 osd.11 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:21:03.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:03.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:03.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:03.794 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:03.794 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:03.794 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:03.794 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:03 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:03.485+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:04.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:04.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:04.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:04.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:04.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:04.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:04.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:04 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:04.514+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:05.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:05.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:05.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:05.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:05.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:05.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:05.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:05 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:05.530+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:07.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:07.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:07.042 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:07.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:07.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:07.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:07.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:07.043 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:06 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:06.556+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-05-14T17:20:43.035504+0000 front 2024-05-14T17:20:43.035474+0000 (oldest deadline 2024-05-14T17:21:05.835969+0000) 2024-05-14T17:21:07.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-05-14T17:19:38.124121+0000 front 2024-05-14T17:19:38.123945+0000 (oldest deadline 2024-05-14T17:20:02.824070+0000) 2024-05-14T17:21:07.792 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-05-14T17:19:48.026338+0000 front 2024-05-14T17:19:48.026122+0000 (oldest deadline 2024-05-14T17:20:13.326119+0000) 2024-05-14T17:21:07.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T17:19:57.828117+0000 front 2024-05-14T17:19:57.828082+0000 (oldest deadline 2024-05-14T17:20:19.528229+0000) 2024-05-14T17:21:07.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-05-14T17:20:05.429259+0000 front 2024-05-14T17:20:05.429128+0000 (oldest deadline 2024-05-14T17:20:27.129184+0000) 2024-05-14T17:21:07.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-05-14T17:20:12.430091+0000 front 2024-05-14T17:20:12.430184+0000 (oldest deadline 2024-05-14T17:20:36.529842+0000) 2024-05-14T17:21:07.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T17:20:20.030677+0000 front 2024-05-14T17:20:20.030717+0000 (oldest deadline 2024-05-14T17:20:43.530665+0000) 2024-05-14T17:21:07.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-05-14T17:20:34.934386+0000 front 2024-05-14T17:20:34.934338+0000 (oldest deadline 2024-05-14T17:21:00.834148+0000) 2024-05-14T17:21:07.793 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:07 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11[61982]: 2024-05-14T17:21:07.529+0000 7fb14ccb9700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-05-14T17:20:43.035504+0000 front 2024-05-14T17:20:43.035474+0000 (oldest deadline 2024-05-14T17:21:05.835969+0000) 2024-05-14T17:21:09.019 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:08 smithi193 podman[76423]: 2024-05-14 17:21:08.729173538 +0000 UTC m=+5.475260422 container died fc020acaf9c94528ce7441f49c995ff17f93e770eb6ebb294ead00a8130d3b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS) 2024-05-14T17:21:09.292 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76423]: 2024-05-14 17:21:09.020374508 +0000 UTC m=+5.766461398 container cleanup fc020acaf9c94528ce7441f49c995ff17f93e770eb6ebb294ead00a8130d3b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:21:09.292 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 bash[76423]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11 2024-05-14T17:21:09.292 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76440]: 2024-05-14 17:21:09.26308338 +0000 UTC m=+0.533300465 container remove fc020acaf9c94528ce7441f49c995ff17f93e770eb6ebb294ead00a8130d3b79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-14T17:21:09.848 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76526]: 2024-05-14 17:21:09.497393088 +0000 UTC m=+0.017305025 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:21:09.848 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76526]: 2024-05-14 17:21:09.597242869 +0000 UTC m=+0.117154789 container create 75a8e9e68738669e02c602ad8325924bdb03ebbc14e440ad150da083fb5e2920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:21:09.848 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76526]: 2024-05-14 17:21:09.774544165 +0000 UTC m=+0.294456092 container init 75a8e9e68738669e02c602ad8325924bdb03ebbc14e440ad150da083fb5e2920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-14T17:21:09.848 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76526]: 2024-05-14 17:21:09.777551919 +0000 UTC m=+0.297463856 container start 75a8e9e68738669e02c602ad8325924bdb03ebbc14e440ad150da083fb5e2920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-14T17:21:09.849 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:09 smithi193 podman[76526]: 2024-05-14 17:21:09.849537296 +0000 UTC m=+0.369449234 container attach 75a8e9e68738669e02c602ad8325924bdb03ebbc14e440ad150da083fb5e2920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-14T17:21:10.292 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:10 smithi193 podman[76526]: 2024-05-14 17:21:10.067224782 +0000 UTC m=+0.587136711 container died 75a8e9e68738669e02c602ad8325924bdb03ebbc14e440ad150da083fb5e2920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:21:10.546 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.11.service' 2024-05-14T17:21:10.568 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:10 smithi193 podman[76552]: 2024-05-14 17:21:10.527741846 +0000 UTC m=+0.454402985 container remove 75a8e9e68738669e02c602ad8325924bdb03ebbc14e440ad150da083fb5e2920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:21:10.569 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:10 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-14T17:21:10.569 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:10 smithi193 systemd[1]: Stopped Ceph osd.11 for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:21:10.569 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 17:21:10 smithi193 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@osd.11.service: Consumed 42.424s CPU time. 2024-05-14T17:21:10.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:10.584 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-14T17:21:10.584 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-14T17:21:10.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.a 2024-05-14T17:21:10.882 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 17:21:10 smithi099 systemd[1]: Stopping Ceph mds.a for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:11.264 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 17:21:10 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a[71583]: 2024-05-14T17:21:10.882+0000 7fb51dd84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:11.264 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 17:21:10 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a[71583]: 2024-05-14T17:21:10.882+0000 7fb51dd84700 -1 mds.a *** got signal Terminated *** 2024-05-14T17:21:12.131 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 17:21:11 smithi099 podman[107306]: 2024-05-14 17:21:11.87584641 +0000 UTC m=+1.090761079 container died bf58256064fadc1e27a94963e9e2cb0e9340602a8a0da6eaacde7da5710376f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, io.buildah.version=1.33.5) 2024-05-14T17:21:12.473 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 17:21:12 smithi099 podman[107306]: 2024-05-14 17:21:12.232362784 +0000 UTC m=+1.447277475 container cleanup bf58256064fadc1e27a94963e9e2cb0e9340602a8a0da6eaacde7da5710376f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:21:12.474 INFO:journalctl@ceph.mds.a.smithi099.stdout:May 14 17:21:12 smithi099 bash[107306]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-a 2024-05-14T17:21:12.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.a.service' 2024-05-14T17:21:12.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:12.640 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-14T17:21:12.640 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-14T17:21:12.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.d 2024-05-14T17:21:13.014 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:12 smithi099 systemd[1]: Stopping Ceph mds.d for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:13.014 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d[70453]: 2024-05-14T17:21:12.961+0000 7f771510b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:13.014 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:12 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d[70453]: 2024-05-14T17:21:12.961+0000 7f771510b700 -1 mds.d *** got signal Terminated *** 2024-05-14T17:21:16.192 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:15 smithi099 podman[107430]: 2024-05-14 17:21:15.871740329 +0000 UTC m=+3.001378556 container died d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-14T17:21:16.514 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:16 smithi099 podman[107430]: 2024-05-14 17:21:16.194174771 +0000 UTC m=+3.323812985 container cleanup d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:21:16.514 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:16 smithi099 bash[107430]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d 2024-05-14T17:21:16.514 INFO:journalctl@ceph.mds.d.smithi099.stdout:May 14 17:21:16 smithi099 podman[107441]: 2024-05-14 17:21:16.378697987 +0000 UTC m=+0.505351062 container remove d9410889b826293ccc259aad1a0eddd59c41f858b972e4ac077754d97a8d80bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2) 2024-05-14T17:21:16.620 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.d.service' 2024-05-14T17:21:16.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:16.654 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-14T17:21:16.654 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-14T17:21:16.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.g 2024-05-14T17:21:16.963 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:16 smithi099 systemd[1]: Stopping Ceph mds.g for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:17.264 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g[71274]: 2024-05-14T17:21:16.962+0000 7f9b0ca64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:17.264 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:16 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g[71274]: 2024-05-14T17:21:16.963+0000 7f9b0ca64700 -1 mds.g *** got signal Terminated *** 2024-05-14T17:21:21.264 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:20 smithi099 podman[107551]: 2024-05-14 17:21:20.979157965 +0000 UTC m=+4.108051647 container died c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T17:21:21.984 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:21 smithi099 podman[107551]: 2024-05-14 17:21:21.630664233 +0000 UTC m=+4.759557937 container cleanup c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T17:21:21.985 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:21 smithi099 bash[107551]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g 2024-05-14T17:21:21.985 INFO:journalctl@ceph.mds.g.smithi099.stdout:May 14 17:21:21 smithi099 podman[107565]: 2024-05-14 17:21:21.79060962 +0000 UTC m=+0.813927488 container remove c7a64b7273422b5aba9167aac6a10b4ac064dcf5e26573165cfdd710fb533294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:21:22.000 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.g.service' 2024-05-14T17:21:22.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:22.032 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-14T17:21:22.033 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-14T17:21:22.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.j 2024-05-14T17:21:22.264 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:22 smithi099 systemd[1]: Stopping Ceph mds.j for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:22.764 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j[70038]: 2024-05-14T17:21:22.341+0000 7fd317a33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:22.764 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:22 smithi099 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j[70038]: 2024-05-14T17:21:22.341+0000 7fd317a33700 -1 mds.j *** got signal Terminated *** 2024-05-14T17:21:24.733 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:24 smithi099 podman[107670]: 2024-05-14 17:21:24.443711867 +0000 UTC m=+2.191537180 container died 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:21:25.014 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:24 smithi099 podman[107670]: 2024-05-14 17:21:24.809262766 +0000 UTC m=+2.557088077 container cleanup 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:21:25.014 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:24 smithi099 bash[107670]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j 2024-05-14T17:21:25.014 INFO:journalctl@ceph.mds.j.smithi099.stdout:May 14 17:21:24 smithi099 podman[107682]: 2024-05-14 17:21:24.960301945 +0000 UTC m=+0.511110532 container remove 19663200bb6274cb69def82131f624b54a975e0b16f2e4d88015be5523821bee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-14T17:21:25.176 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.j.service' 2024-05-14T17:21:25.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:25.209 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-14T17:21:25.209 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-14T17:21:25.209 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.b 2024-05-14T17:21:25.492 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:21:25 smithi150 systemd[1]: Stopping Ceph mds.b for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:25.760 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:21:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b[66605]: 2024-05-14T17:21:25.493+0000 7f183b31c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:25.761 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:21:25 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b[66605]: 2024-05-14T17:21:25.493+0000 7f183b31c700 -1 mds.b *** got signal Terminated *** 2024-05-14T17:21:26.953 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:21:26 smithi150 podman[74870]: 2024-05-14 17:21:26.906349435 +0000 UTC m=+1.508175830 container died d5eb5f5c43f62ec987a2b0de7714fd5d366bb35c71e5ad16fa7fe48e552cec85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD) 2024-05-14T17:21:27.443 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:21:27 smithi150 podman[74870]: 2024-05-14 17:21:27.235020087 +0000 UTC m=+1.836846478 container cleanup d5eb5f5c43f62ec987a2b0de7714fd5d366bb35c71e5ad16fa7fe48e552cec85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-14T17:21:27.443 INFO:journalctl@ceph.mds.b.smithi150.stdout:May 14 17:21:27 smithi150 bash[74870]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-b 2024-05-14T17:21:27.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.b.service' 2024-05-14T17:21:27.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:27.691 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-14T17:21:27.691 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-14T17:21:27.691 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.e 2024-05-14T17:21:28.005 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 17:21:27 smithi150 systemd[1]: Stopping Ceph mds.e for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:28.453 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 17:21:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e[65987]: 2024-05-14T17:21:28.005+0000 7f8beaf05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:28.453 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 17:21:28 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e[65987]: 2024-05-14T17:21:28.005+0000 7f8beaf05700 -1 mds.e *** got signal Terminated *** 2024-05-14T17:21:30.838 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 17:21:30 smithi150 podman[75003]: 2024-05-14 17:21:30.503227148 +0000 UTC m=+2.595661374 container died c1b79095d2af9e2d651964d9f006c7af2ebb343e064c9e070440ec3135f7de7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-14T17:21:31.096 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 17:21:30 smithi150 podman[75003]: 2024-05-14 17:21:30.930477484 +0000 UTC m=+3.022911738 container cleanup c1b79095d2af9e2d651964d9f006c7af2ebb343e064c9e070440ec3135f7de7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-14T17:21:31.097 INFO:journalctl@ceph.mds.e.smithi150.stdout:May 14 17:21:30 smithi150 bash[75003]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-e 2024-05-14T17:21:31.317 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.e.service' 2024-05-14T17:21:31.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:31.350 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-14T17:21:31.350 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-14T17:21:31.350 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.h 2024-05-14T17:21:31.666 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:31 smithi150 systemd[1]: Stopping Ceph mds.h for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:31.952 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h[66289]: 2024-05-14T17:21:31.667+0000 7fa8d2a27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:31.953 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:31 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h[66289]: 2024-05-14T17:21:31.667+0000 7fa8d2a27700 -1 mds.h *** got signal Terminated *** 2024-05-14T17:21:35.203 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:34 smithi150 podman[75124]: 2024-05-14 17:21:34.912312669 +0000 UTC m=+3.346096375 container died 2dd5281220ac185d4bf798f1ff48f78f1489710275e7873fb506170261cbb467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-14T17:21:35.650 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:35 smithi150 podman[75124]: 2024-05-14 17:21:35.242595439 +0000 UTC m=+3.676379137 container cleanup 2dd5281220ac185d4bf798f1ff48f78f1489710275e7873fb506170261cbb467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-14T17:21:35.650 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:35 smithi150 bash[75124]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h 2024-05-14T17:21:35.650 INFO:journalctl@ceph.mds.h.smithi150.stdout:May 14 17:21:35 smithi150 podman[75137]: 2024-05-14 17:21:35.427132737 +0000 UTC m=+0.515040993 container remove 2dd5281220ac185d4bf798f1ff48f78f1489710275e7873fb506170261cbb467 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:21:35.669 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.h.service' 2024-05-14T17:21:35.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:35.701 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-14T17:21:35.701 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-14T17:21:35.702 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.k 2024-05-14T17:21:35.952 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:35 smithi150 systemd[1]: Stopping Ceph mds.k for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:36.453 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k[65673]: 2024-05-14T17:21:36.037+0000 7f7df1f7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:36.453 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:36 smithi150 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k[65673]: 2024-05-14T17:21:36.037+0000 7f7df1f7f700 -1 mds.k *** got signal Terminated *** 2024-05-14T17:21:39.854 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:39 smithi150 podman[75245]: 2024-05-14 17:21:39.600384947 +0000 UTC m=+3.683925606 container died 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:21:40.137 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:39 smithi150 podman[75245]: 2024-05-14 17:21:39.946249652 +0000 UTC m=+4.029790304 container cleanup 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:21:40.138 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:39 smithi150 bash[75245]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k 2024-05-14T17:21:40.373 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.k.service' 2024-05-14T17:21:40.393 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:40 smithi150 podman[75259]: 2024-05-14 17:21:40.138773929 +0000 UTC m=+0.534628328 container remove 2b9040597b5e474a17dab922d80f9d5ea5b28f48acce794a4a5cbabcb3a0d39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514) 2024-05-14T17:21:40.393 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:40 smithi150 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.k.service: Deactivated successfully. 2024-05-14T17:21:40.393 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:40 smithi150 systemd[1]: Stopped Ceph mds.k for 2a006d80-1212-11ef-bc98-c7b262605968. 2024-05-14T17:21:40.393 INFO:journalctl@ceph.mds.k.smithi150.stdout:May 14 17:21:40 smithi150 systemd[1]: ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.k.service: Consumed 1.167s CPU time. 2024-05-14T17:21:40.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:40.404 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-14T17:21:40.404 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-14T17:21:40.404 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.c 2024-05-14T17:21:40.693 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:21:40 smithi193 systemd[1]: Stopping Ceph mds.c for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:40.962 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:21:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c[67374]: 2024-05-14T17:21:40.694+0000 7f8d5e63c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:40.963 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:21:40 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c[67374]: 2024-05-14T17:21:40.694+0000 7f8d5e63c700 -1 mds.c *** got signal Terminated *** 2024-05-14T17:21:43.243 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:21:42 smithi193 podman[76633]: 2024-05-14 17:21:42.915587288 +0000 UTC m=+2.317851012 container died d455cbdc36f6ec19e65b40187b0fe61b205a113a1f957904a2b0925fb9f8e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:21:43.542 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:21:43 smithi193 podman[76633]: 2024-05-14 17:21:43.319055972 +0000 UTC m=+2.721319686 container cleanup d455cbdc36f6ec19e65b40187b0fe61b205a113a1f957904a2b0925fb9f8e70c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:21:43.542 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 17:21:43 smithi193 bash[76633]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-c 2024-05-14T17:21:43.685 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.c.service' 2024-05-14T17:21:43.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:43.720 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-14T17:21:43.720 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-14T17:21:43.720 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f 2024-05-14T17:21:44.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:21:43 smithi193 systemd[1]: Stopping Ceph mds.f for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:44.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:21:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: 2024-05-14T17:21:44.030+0000 7fd196957700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:44.292 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:21:44 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f[72777]: 2024-05-14T17:21:44.030+0000 7fd196957700 -1 mds.f *** got signal Terminated *** 2024-05-14T17:21:47.197 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:21:46 smithi193 podman[76767]: 2024-05-14 17:21:46.9252593 +0000 UTC m=+2.979621506 container died 4114072b90bd0db91612eca1c5c33754a152e0f1d738a05ebfd930bea12f1b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:21:47.542 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:21:47 smithi193 podman[76767]: 2024-05-14 17:21:47.28118024 +0000 UTC m=+3.335542433 container cleanup 4114072b90bd0db91612eca1c5c33754a152e0f1d738a05ebfd930bea12f1b00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD) 2024-05-14T17:21:47.542 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 17:21:47 smithi193 bash[76767]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-f 2024-05-14T17:21:47.716 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.f.service' 2024-05-14T17:21:47.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:47.753 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-14T17:21:47.753 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-14T17:21:47.754 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.i 2024-05-14T17:21:48.292 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:21:47 smithi193 systemd[1]: Stopping Ceph mds.i for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:48.292 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:21:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: 2024-05-14T17:21:48.059+0000 7f0be9d24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:48.292 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:21:48 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i[67690]: 2024-05-14T17:21:48.059+0000 7f0be9d24700 -1 mds.i *** got signal Terminated *** 2024-05-14T17:21:51.267 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:21:50 smithi193 podman[76888]: 2024-05-14 17:21:50.910784873 +0000 UTC m=+2.932928582 container died 3d536fb5be4615227372857796d43d297f6cb82f524e6dfbafb05bf8834494af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T17:21:51.542 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:21:51 smithi193 podman[76888]: 2024-05-14 17:21:51.343226835 +0000 UTC m=+3.365370552 container cleanup 3d536fb5be4615227372857796d43d297f6cb82f524e6dfbafb05bf8834494af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:21:51.542 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 17:21:51 smithi193 bash[76888]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-i 2024-05-14T17:21:51.764 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.i.service' 2024-05-14T17:21:51.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:51.795 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-14T17:21:51.796 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-14T17:21:51.796 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.l 2024-05-14T17:21:52.204 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:21:51 smithi193 systemd[1]: Stopping Ceph mds.l for 2a006d80-1212-11ef-bc98-c7b262605968... 2024-05-14T17:21:52.542 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:21:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l[66763]: 2024-05-14T17:21:52.204+0000 7f1bd1aa5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:21:52.542 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:21:52 smithi193 ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l[66763]: 2024-05-14T17:21:52.204+0000 7f1bd1aa5700 -1 mds.l *** got signal Terminated *** 2024-05-14T17:21:55.288 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:21:54 smithi193 podman[77008]: 2024-05-14 17:21:54.91840699 +0000 UTC m=+2.898576397 container died 291487fb2beb0c51269a54362b0064a46dae52a9041599588367e3734f57b3dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:21:55.542 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:21:55 smithi193 podman[77008]: 2024-05-14 17:21:55.372060409 +0000 UTC m=+3.352229809 container cleanup 291487fb2beb0c51269a54362b0064a46dae52a9041599588367e3734f57b3dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-14T17:21:55.542 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 17:21:55 smithi193 bash[77008]: ceph-2a006d80-1212-11ef-bc98-c7b262605968-mds-l 2024-05-14T17:21:55.754 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a006d80-1212-11ef-bc98-c7b262605968@mds.l.service' 2024-05-14T17:21:55.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:21:55.784 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-14T17:21:55.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:21:55.976 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:21:58.436 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:21:58.627 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:22:01.163 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:22:01.352 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:22:03.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:22:03.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:22:03.747 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:22:03.773 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-14T17:22:03.776 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi099/crash 2024-05-14T17:22:03.777 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:22:03.814 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:22:03.815 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:22:03.817 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi150/crash 2024-05-14T17:22:03.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:22:03.855 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:22:03.855 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:22:03.857 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi193/crash 2024-05-14T17:22:03.858 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:22:03.897 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/2a006d80-1212-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:22:03.898 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:22:03.899 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-14T17:22:03.899 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-14T17:22:03.935 INFO:tasks.cephadm:Compressing logs... 2024-05-14T17:22:03.935 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:22:03.978 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:22:03.980 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:22:04.002 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:22:04.002 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2024-05-14T17:22:04.002 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:22:04.002 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-14T17:22:04.005 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.j.log: /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:22:04.006 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-14T17:22:04.016 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-14T17:22:04.017 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-14T17:22:04.017 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-14T17:22:04.017 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-14T17:22:04.017 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.z.log: 87.3% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:22:04.017 INFO:teuthology.orchestra.run.smithi193.stderr: 90.5% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:22:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.b.log: /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-14T17:22:04.025 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.h.log: 87.3% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 90.5% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:22:04.026 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-14T17:22:04.027 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-14T17:22:04.027 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log: 92.2% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-14T17:22:04.027 INFO:teuthology.orchestra.run.smithi193.stderr: 92.3% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-14T17:22:04.027 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:22:04.028 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-14T17:22:04.028 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-14T17:22:04.028 INFO:teuthology.orchestra.run.smithi193.stderr: 84.0% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:22:04.028 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:22:04.028 INFO:teuthology.orchestra.run.smithi099.stderr: 94.7% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr: 94.6% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.a.log: 84.3% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-14T17:22:04.029 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-14T17:22:04.030 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:22:04.030 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-14T17:22:04.030 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:22:04.030 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:22:04.031 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-14T17:22:04.032 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-14T17:22:04.033 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:22:04.033 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:22:04.038 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-14T17:22:04.041 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:22:04.051 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:22:04.058 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.l.log: 91.0%/var/log/ceph/cephadm.log: -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:22:04.073 INFO:teuthology.orchestra.run.smithi193.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:22:04.451 INFO:teuthology.orchestra.run.smithi099.stderr: 90.2% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-14T17:22:04.511 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-14T17:22:05.054 INFO:teuthology.orchestra.run.smithi193.stderr: 92.7% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-14T17:22:05.510 INFO:teuthology.orchestra.run.smithi099.stderr: 90.4% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-14T17:22:05.766 INFO:teuthology.orchestra.run.smithi193.stderr: 95.4% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-14T17:22:06.169 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-14T17:22:07.598 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-14T17:22:07.843 INFO:teuthology.orchestra.run.smithi099.stderr: 93.7% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-14T17:22:07.851 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-14T17:22:08.330 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-14T17:22:08.672 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-14T17:22:09.024 INFO:teuthology.orchestra.run.smithi099.stderr: 93.8% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-14T17:22:09.026 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-05-14T17:22:09.027 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m5.035s 2024-05-14T17:22:09.027 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m18.068s 2024-05-14T17:22:09.027 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.699s 2024-05-14T17:22:09.079 INFO:teuthology.orchestra.run.smithi150.stderr: 93.8% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-14T17:22:09.672 INFO:teuthology.orchestra.run.smithi193.stderr: 95.4% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-14T17:22:09.942 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-14T17:22:10.008 INFO:teuthology.orchestra.run.smithi193.stderr: 94.1% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-14T17:22:10.029 INFO:teuthology.orchestra.run.smithi193.stderr: 94.2% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-14T17:22:10.760 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-14T17:22:11.330 INFO:teuthology.orchestra.run.smithi193.stderr: 94.2% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-14T17:22:26.444 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-14T17:22:30.997 INFO:teuthology.orchestra.run.smithi150.stderr: 93.8% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-14T17:22:30.999 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-05-14T17:22:30.999 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m27.005s 2024-05-14T17:22:30.999 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m48.312s 2024-05-14T17:22:30.999 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m1.637s 2024-05-14T17:23:22.163 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/2a006d80-1212-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-14T17:23:22.165 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-05-14T17:23:22.165 INFO:teuthology.orchestra.run.smithi193.stderr:real 1m18.170s 2024-05-14T17:23:22.165 INFO:teuthology.orchestra.run.smithi193.stderr:user 2m8.895s 2024-05-14T17:23:22.165 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m4.159s 2024-05-14T17:23:22.166 INFO:tasks.cephadm:Archiving logs... 2024-05-14T17:23:22.170 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi099/log 2024-05-14T17:23:22.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:23:24.723 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi150/log 2024-05-14T17:23:24.724 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:23:31.405 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi193/log 2024-05-14T17:23:31.406 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:23:48.848 INFO:tasks.cephadm:Removing cluster... 2024-05-14T17:23:48.849 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --force 2024-05-14T17:23:49.044 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:23:49.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --force 2024-05-14T17:23:49.511 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:23:49.815 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a006d80-1212-11ef-bc98-c7b262605968 --force 2024-05-14T17:23:50.007 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 2a006d80-1212-11ef-bc98-c7b262605968 2024-05-14T17:23:50.409 INFO:tasks.cephadm:Removing cephadm ... 2024-05-14T17:23:50.409 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:23:50.428 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:23:50.445 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:23:50.460 INFO:tasks.cephadm:Teardown complete 2024-05-14T17:23:50.461 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-14T17:23:50.476 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-14T17:23:50.476 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:23:50.479 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:23:50.487 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:23:50.594 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-14T17:23:50.605 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-14T17:23:50.605 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:23:50.608 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:23:50.610 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:23:50.622 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:23:50.623 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:23:50.624 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:23:50.626 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:23:50.626 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-14T17:23:50.626 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +726us[ +728us] +/- 46ms 2024-05-14T17:23:50.626 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1044us[-1043us] +/- 80ms 2024-05-14T17:23:50.627 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -217us[ -216us] +/- 70ms 2024-05-14T17:23:50.627 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:23:50.628 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:23:50.629 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-05-14T17:23:50.629 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 7 377 129 +737us[ +772us] +/- 45ms 2024-05-14T17:23:50.629 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1004us[-1004us] +/- 80ms 2024-05-14T17:23:50.629 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 -176us[ -176us] +/- 71ms 2024-05-14T17:23:50.629 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:23:50.631 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:23:50.631 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-05-14T17:23:50.631 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 7 377 0 +818us[ +812us] +/- 47ms 2024-05-14T17:23:50.631 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 65 -976us[ -984us] +/- 79ms 2024-05-14T17:23:50.631 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 134 -112us[ -109us] +/- 69ms 2024-05-14T17:23:50.631 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:23:50.632 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-14T17:23:50.642 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-14T17:23:50.643 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-14T17:23:50.653 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:23:50.755 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:23:50.821 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:23:50.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:23:50.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:50.916 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:23:50.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:50.942 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:23:50.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:50.969 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-14T17:23:50.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-14T17:23:50.993 INFO:teuthology.task.internal:Duration was 2433.154672 seconds 2024-05-14T17:23:50.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-14T17:23:51.004 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-14T17:23:51.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:23:51.007 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:23:51.009 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:23:51.042 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:23:51.045 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:23:51.045 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:23:51.495 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-14T17:23:51.495 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-05-14T17:23:51.496 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:23:51.518 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-05-14T17:23:51.518 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:23:51.566 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-05-14T17:23:51.566 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:23:51.590 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-14T17:23:51.590 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:23:51.593 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:23:51.609 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:23:51.655 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-14T17:23:51.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:23:51.660 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:23:51.673 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:23:52.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-14T17:23:52.379 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-14T17:23:52.379 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:23:52.405 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:23:52.429 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:23:52.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-14T17:23:52.465 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:52.468 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:52.472 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:52.489 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-05-14T17:23:52.492 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-05-14T17:23:52.516 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-05-14T17:23:52.543 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:52.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:52.558 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:52.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:52.576 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:52.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:52.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-14T17:23:52.610 INFO:teuthology.task.internal:Transferring archived files... 2024-05-14T17:23:52.611 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi099 2024-05-14T17:23:52.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:23:52.665 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi150 2024-05-14T17:23:52.665 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:23:52.700 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182/remote/smithi193 2024-05-14T17:23:52.700 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:23:52.735 INFO:teuthology.task.internal:Removing archive directory... 2024-05-14T17:23:52.735 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:23:52.737 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:23:52.740 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:23:52.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-14T17:23:52.795 INFO:teuthology.task.internal:Not uploading archives. 2024-05-14T17:23:52.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-14T17:23:52.839 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-14T17:23:52.840 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:23:52.842 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:23:52.845 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:23:52.855 INFO:teuthology.orchestra.run.smithi099.stdout: 265585 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:23 /home/ubuntu/cephtest 2024-05-14T17:23:52.857 INFO:teuthology.orchestra.run.smithi150.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:23 /home/ubuntu/cephtest 2024-05-14T17:23:52.859 INFO:teuthology.orchestra.run.smithi193.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:23 /home/ubuntu/cephtest 2024-05-14T17:23:52.860 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-14T17:23:52.871 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-14T17:23:52.895 INFO:teuthology.nuke:Checking targets against current locks 2024-05-14T17:23:52.920 DEBUG:teuthology.nuke:shortname: smithi099 2024-05-14T17:23:52.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:23:52.944 DEBUG:teuthology.nuke:shortname: smithi150 2024-05-14T17:23:52.944 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:23:52.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:34:57.919365', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBiw6ukUsbpFXoMLy1it/jvYB0cNHolqhaDjCzxMXmctvBbEDA3k22Et5mgpxYNkmHJt8KTUH6A6OBjv/UficvI='} 2024-05-14T17:23:52.966 DEBUG:teuthology.nuke:shortname: smithi193 2024-05-14T17:23:52.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:23:52.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:34:57.920400', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFnPtW/ydXNhKazr9+bslerCnNDZ5m0xnt08qKhiVkvrSm3V2YUkJdQZklyFK/ElaJDaEuFwwBpQSfc84DzK6mk='} 2024-05-14T17:23:52.987 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:34:57.917640', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ThY9+JfMC+UiF7NZfMnkvWr82I2LnuWBPu2xxg+HGJQTXUm953LGwGKC9NOe7SX7shQkPL15XJ+rsvh0BTiI='} 2024-05-14T17:23:53.000 INFO:teuthology.orchestra.console.smithi099:Power off 2024-05-14T17:23:53.001 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:23:53.030 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2024-05-14T17:23:53.030 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:53.049 INFO:teuthology.orchestra.console.smithi193:Power off 2024-05-14T17:23:53.050 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:23:53.065 INFO:teuthology.orchestra.console.smithi150:Power off 2024-05-14T17:23:53.065 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:23:53.076 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-05-14T17:23:53.076 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-05-14T17:23:53.077 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:53.091 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-05-14T17:23:53.092 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:53.101 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-14T17:23:53.120 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-05-14T17:23:57.077 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:57.102 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:57.121 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:57.188 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-05-14T17:23:57.213 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-14T17:23:57.229 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-05-14T17:24:01.193 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:24:01.214 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:24:01.230 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:24:01.304 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2024-05-14T17:24:01.323 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-14T17:24:01.339 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-05-14T17:24:01.405 INFO:teuthology.orchestra.console.smithi099:Power off completed 2024-05-14T17:24:01.440 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-05-14T17:24:05.324 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:24:05.439 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-05-14T17:24:05.540 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-05-14T17:24:05.646 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2433.1546716690063 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi099 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9e0ae7c57d244ea58049d58556d61f3b status: fail success: false 2024-05-14T17:24:05.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T17:24:05.757 INFO:teuthology.run:FAIL